-- Logs begin at Thu 2022-05-19 05:41:07 UTC, end at Thu 2022-05-19 06:52:41 UTC. -- May 19 06:14:40.284134 dv-cinder-836996-22-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 06:14:40.292891 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 06:14:40.296337 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 06:14:40 2022] *** May 19 06:14:40.296337 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 06:14:40.296337 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 06:14:40.296337 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: nodename: dv-cinder-836996-22-iscsi May 19 06:14:40.296337 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: machine: x86_64 May 19 06:14:40.296337 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: clock source: unix May 19 06:14:40.296337 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: pcre jit disabled May 19 06:14:40.296337 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: detected number of CPU cores: 4 May 19 06:14:40.296337 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: current working directory: / May 19 06:14:40.296337 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: detected binary path: /usr/bin/uwsgi-core May 19 06:14:40.296337 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: your processes number limit is 47680 May 19 06:14:40.296337 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: your memory page size is 4096 bytes May 19 06:14:40.296337 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: detected max file descriptor number: 2048 May 19 06:14:40.296337 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: lock engine: pthread robust mutexes May 19 06:14:40.297115 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: thunder lock: enabled May 19 06:14:40.297115 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 06:14:40.297115 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 06:14:40.348470 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: Python main interpreter initialized at 0x564f0f7fb9c0 May 19 06:14:40.348470 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: python threads support enabled May 19 06:14:40.348470 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: your server socket listen backlog is limited to 100 connections May 19 06:14:40.348470 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: your mercy for graceful operations on workers is 90 seconds May 19 06:14:40.348874 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: mapped 403077 bytes (393 KB) for 2 cores May 19 06:14:40.348952 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: *** Operational MODE: preforking *** May 19 06:14:40.349272 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: *** uWSGI is running in multiple interpreter mode *** May 19 06:14:40.349272 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: spawned uWSGI master process (pid: 80269) May 19 06:14:40.349520 dv-cinder-836996-22-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 06:14:40.350662 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: spawned uWSGI worker 1 (pid: 80280, cores: 1) May 19 06:14:40.350662 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: spawned uWSGI worker 2 (pid: 80281, cores: 1) May 19 06:14:40.350662 dv-cinder-836996-22-iscsi devstack@keystone.service[80269]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 06:14:46.212459 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 06:14:46.212459 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: from cryptography.utils import int_from_bytes May 19 06:14:46.213340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 06:14:46.213340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: from cryptography.utils import int_from_bytes May 19 06:14:46.263994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 06:14:46.264220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 06:14:46.264374 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] command line args: [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 06:14:46.264650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 06:14:46.264832 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ================================================================================ {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 06:14:46.265092 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] admin_token = **** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.265372 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] config_dir = [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.265567 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.265752 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] config_source = [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.265949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] debug = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.266162 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.266578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.266757 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.267019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.267273 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.267509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] list_limit = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.267693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] log_config_append = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.267878 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.268053 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] log_dir = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.268234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] log_file = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.268474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.268726 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.268977 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.269261 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.269606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.269606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.269870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.270060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.270300 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.270537 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.270774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.271011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.271264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.271542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.271795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.272306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.272494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] publish_errors = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.272733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.272967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.273156 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.273430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.273671 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.273910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] standard_threads = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.274145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.274391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.274572 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.274751 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] use_journal = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.274930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] use_json = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.275110 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] use_stderr = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.275287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] use_syslog = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.275521 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.275786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.276027 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.276291 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.276532 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.276783 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.277025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.277273 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.277533 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.277780 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.278030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.278288 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.278525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.278776 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.279016 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.279264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.279514 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.279753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.280010 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.280266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.280514 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.281005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.281258 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.281555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.282313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.282577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.282812 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.283050 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.283285 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.283522 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.283775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.284025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.284258 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.284492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.284739 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.284987 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.285228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.285508 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.285745 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.285985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.286226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.286462 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.286697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.286932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.287165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.287406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.287660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.287896 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.288139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.288374 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.288615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.288854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.289105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.289362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.289606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.289849 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.290089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.290330 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.290571 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.290806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.291058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.291298 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.291552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.291785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.292060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.292302 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.292536 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.292777 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.293021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.293335 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.293558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.293808 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.294061 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.294302 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.294542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.294782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.295032 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.295274 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.295515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.295760 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.296001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.296239 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.296475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.296715 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.296954 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.297217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.298134 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.298365 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.298609 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.298856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.299099 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.299345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.299581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.299816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.300241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.300482 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.300721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.300956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.301203 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.301468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.301724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.301967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.302373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.302640 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.302889 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.303128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] auth.external = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.303369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.303671 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.303909 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.304147 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] auth.password = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.304383 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] auth.token = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.304628 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.304868 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.305119 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.305380 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.305634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.305885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.306120 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.306359 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] credential.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.306609 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.306860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.307112 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.307353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.307601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.307839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.308103 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.308361 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.308601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.308857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.309149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.309430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.309698 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.309945 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.310205 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.310443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] federation.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.310684 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.310934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.311182 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.311415 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.311665 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.311905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.312163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.312405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.312663 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.312904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.313197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.313491 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.313713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.313967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.314361 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.314512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.314661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.314894 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.315137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.315389 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.315646 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.315880 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.316116 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.316348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.316514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 06:14:46.316514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: from cryptography.utils import int_from_bytes May 19 06:14:46.317237 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.317237 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.317532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 06:14:46.317532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: from cryptography.utils import int_from_bytes May 19 06:14:46.317706 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.317706 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.317706 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.317960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.318054 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.318296 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.318530 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.318769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.319014 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.319255 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.319502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.319744 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.319995 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.320228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.320479 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.320726 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.320968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.321218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.321493 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.321727 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.321967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.322201 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.322442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.322683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.322931 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.323176 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.323416 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.323668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.323917 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.324250 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.324412 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.324607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.324859 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.325096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.325353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.325595 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.325826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.326068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.326327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.326562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.326812 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.327060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.327309 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.327544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.327777 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.328012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.328254 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.328490 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.329095 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.329095 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.329619 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.329619 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.329879 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.330088 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.330328 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.330581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.330823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.331074 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.331290 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.331542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.331790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.332044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.332282 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.332538 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.332773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.333021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.333266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.333528 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.333769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.334016 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.334261 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.334496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.334748 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.334992 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] resource.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.335244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.335494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.335734 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.336457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.336457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.336457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.336786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.336894 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.337137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.337398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] role.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.337646 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] role.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.337881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.338133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.338383 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.338654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.338870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.339116 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.339365 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.339613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.339862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.340097 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.340344 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.340594 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.340844 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.341092 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.341552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.341792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.342043 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.342294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.342543 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.342792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.343030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.343290 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.343543 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.343791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.344026 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.344262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.345190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.345588 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.345858 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.346120 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.346367 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.346607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.346846 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.347088 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] token.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.347330 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.347584 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.347822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.348082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.348331 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.348573 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.348822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.349069 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.349342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.349585 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.349828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.350066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.350316 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.350568 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.350801 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.351047 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.351349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.351615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.connection = **** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.351864 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.352116 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.352359 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.352603 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.352842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.353089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.353350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.353593 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.353843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.354083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.354332 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.354571 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.354827 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.355069 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.355362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.355564 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.355814 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.356047 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.356220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 06:14:46.370426 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 06:14:46.370702 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 06:14:46.370820 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] command line args: [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 06:14:46.371067 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 06:14:46.371249 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ================================================================================ {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 06:14:46.371508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] admin_token = **** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.371758 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] config_dir = [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.371950 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.372135 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] config_source = [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.372317 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] debug = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.372539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.372781 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.372964 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.373227 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.373512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.373749 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] list_limit = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.373932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] log_config_append = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.374113 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.374293 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] log_dir = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.374474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] log_file = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.374715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.375079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.375335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.376156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.376344 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.377197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.377491 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.377751 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.377995 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.378236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.378478 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.378717 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.378973 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.379225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.379509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.380032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.380221 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] publish_errors = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.380462 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.380704 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.380890 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.381146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.381418 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.381666 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] standard_threads = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.381909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.382163 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.382345 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.382527 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] use_journal = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.382710 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] use_json = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.382892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] use_stderr = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.383074 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] use_syslog = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.383312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 06:14:46.383580 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.383826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.384094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.384342 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.384597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.384842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.385098 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.385366 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.385625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.385878 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.386136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.386376 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.386631 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.386880 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.387133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.387386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.387628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.387889 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.388150 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.388403 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.388586 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.388712 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.388836 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.388951 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.389068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.389195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.389424 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.389683 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.389921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.390157 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.390392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.390633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.390889 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.391148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.392005 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.392911 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.393180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.393469 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.393720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.393979 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.394221 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.394466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.394709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.394945 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.395181 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.395418 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.395656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.395900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.396155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.396393 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.396637 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.396873 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.397115 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.397392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.397647 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.397884 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.398127 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.398369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.398617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.398726 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.398863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.399103 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.399338 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.399607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.399833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.400085 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.400322 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.400588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.400832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.401068 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.401330 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.401577 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.401854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.402097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.402352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.402605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.402848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.403089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.403329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.403584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.403828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.404072 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.404318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.404561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.404804 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.405045 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.405324 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.405551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.406074 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.406321 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.406567 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.406864 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.407664 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.407930 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.408177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.408420 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.408659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.408896 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.409131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.409429 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.409668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.409914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.410155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.410409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.410650 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.410910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.411177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.411423 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.411660 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] auth.external = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.411895 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.412197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.412434 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.412668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] auth.password = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.412913 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] auth.token = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.413157 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.413425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.413679 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.413915 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.414168 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.414422 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.414658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.414898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] credential.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.415149 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.415411 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.415664 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.415906 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.416155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.416395 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.416664 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.416921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.417169 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.417452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.417747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.417999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.418267 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.418512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.418774 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.419013 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] federation.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.419254 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.419504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.419754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.419988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.420241 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.420483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.420742 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.420982 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.421243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.421512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.421766 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.422006 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.422256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.422574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.423471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.423720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.423978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.424221 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.424467 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.424725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.424984 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.425222 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.425487 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.425723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.425974 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.426227 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.426479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.426739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.426990 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.427266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.427515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.427756 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.427880 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.428002 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.428121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.428244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.428440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.428680 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.428927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.429184 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.429465 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.429702 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.429957 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.430209 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.430449 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.430701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.430952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.431189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.431429 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.431665 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.431907 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.432149 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.432398 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.432639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.432881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.433135 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.433415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.433655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.433890 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.434145 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.434398 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.434641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.434882 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.435122 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.435356 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.435601 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.435866 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.436102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.436354 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.436614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.436866 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.437113 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.437373 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.437615 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.437854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.438648 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.439451 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.439720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.439980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.440237 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.440493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.440747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.440983 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.441236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.441509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.441752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.441995 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.442237 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.442487 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.442738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.442979 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.443250 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.443487 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.443741 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.443976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.444218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.444462 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.444716 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.444966 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.445204 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.445474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.445721 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] resource.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.445975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.446226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.446470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.446727 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.446984 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.447224 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.447475 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.447717 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.447965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.448203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] role.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.448452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] role.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.448688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.448941 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.449194 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.449475 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.449729 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.449980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.450232 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.450481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.450733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.450973 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.451226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.451478 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.451729 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.451982 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.452417 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.452658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.452914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.453166 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.453444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.453868 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.455195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.455489 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.455753 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.455999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.456239 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.456479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.456717 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.456974 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.457236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.457519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.457776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.458024 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.458261 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.458505 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] token.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.458748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.459001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.459245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.459507 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.459760 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.460003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.460253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.460503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.460755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.460997 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.461242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.461510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.461763 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.462029 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.462266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.462560 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.462832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.463097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.connection = **** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.463348 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.463601 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.463844 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.464091 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.464331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.464574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.464815 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.465055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.465325 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.465566 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.465813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.466052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.466305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.466545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.466794 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.467028 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.467274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.467514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 06:14:46.467690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 06:14:46.483345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.496922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.497139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.497405 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.508322 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.529209 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.529452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.538867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.539106 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.571913 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.608460 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.608688 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.608892 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.609116 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.609366 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.609587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.612954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.613189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.636333 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.636568 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.636774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.637001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.637223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.637471 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.637690 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.650287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.661575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.661813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.662042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.662266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.662488 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.662711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.665566 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.665803 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.666031 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.666257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.666477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.666693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.666908 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.681179 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.681449 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.681685 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.681920 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.682146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.682369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.682591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.684897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.685115 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 06:14:46.685485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.685693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 06:14:46.685966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.686227 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.686484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.686740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.692274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.695438 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.695674 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.695902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.696124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.696340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.696556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.696770 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 06:14:46.698531 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.698835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.699109 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.699395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.708330 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.708529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.708651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 06:14:46.708921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.709127 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 06:14:46.709431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.709699 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.709978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.710241 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.711696 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.711978 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.712396 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.712731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.712966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.713227 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.713522 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.713798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.714070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.714343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.714611 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.722371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.722668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.722943 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.723249 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.731586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.735545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.736438 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.736848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.737186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.737521 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.737875 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.738157 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.738437 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.738717 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.739001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.739280 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.742320 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.743975 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.744258 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.749606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.749889 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.750168 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.757817 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.758107 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.762795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.763088 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.763374 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.763658 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.763937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.764235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.764490 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.764758 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.768258 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.769987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.770271 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.775712 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.776000 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.776293 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.784895 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.785101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.785270 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.785461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.789873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.790044 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.790193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.790480 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.790764 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.791043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.791334 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.791607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.791895 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.793551 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.793849 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.799523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.803168 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.807031 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.807336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.807622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.807925 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.808261 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.808570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.812239 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.812546 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.818365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.822094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.822395 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.827464 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.828296 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.828726 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.829001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.829266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.831965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.835655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.835965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.836256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.836542 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.836831 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.837120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.837271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.839028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.839310 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.842424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.842703 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.842998 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.843277 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.843567 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.857155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.857436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.857595 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.857747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.857897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.858047 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.858197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.858357 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.858504 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.858651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.858798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.867447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.869176 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.869491 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.872545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.872836 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.873120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.873440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.873722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.878208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.881549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.884886 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.886637 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.888290 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.888609 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.888852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.889031 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.889203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.889484 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.889770 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.890051 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.890331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.890619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.896576 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.898275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.901582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.901880 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.902166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.902442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.911741 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.912230 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.912561 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.912847 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.915346 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.918730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.920371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.920807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.921107 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.921420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.921697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.921973 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.922125 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.922275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.922924 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.923170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.923322 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.923471 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.923709 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.923982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.924262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.924524 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.924795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.925064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.931558 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.933213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.936563 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.936863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.937158 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.937464 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.947946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.948244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.948527 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.957325 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 06:14:46.957618 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.957900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.958181 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.958465 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.958740 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.959015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.959297 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.959571 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.959841 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.960124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.960397 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.960781 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.961100 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 06:14:46.965019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564f0f7fb9c0 pid: 80280 (default app) May 19 06:14:46.968557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6727132-0e6e-430d-9ec9-ab7572086571 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:14:46.968858 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6727132-0e6e-430d-9ec9-ab7572086571 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:14:46.969142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6727132-0e6e-430d-9ec9-ab7572086571 None None] PATH_INFO: `/v3` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:14:46.973794 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1/1] 10.222.0.65 () {50 vars in 791 bytes} [Thu May 19 06:14:46 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:14:47.003149 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564f0f7fb9c0 pid: 80281 (default app) May 19 06:14:55.624495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3595d-456a-44a5-ae02-b2cba217ca33 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:14:55.625093 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3595d-456a-44a5-ae02-b2cba217ca33 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:14:55.625093 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3595d-456a-44a5-ae02-b2cba217ca33 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:14:55.627287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 2/2] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:14:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:14:55.635754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6922f-3194-4d33-ac2e-7304d526eb94 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:14:55.636447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6922f-3194-4d33-ac2e-7304d526eb94 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:14:55.636447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6922f-3194-4d33-ac2e-7304d526eb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:14:55.709214 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: INFO dbcounter [None req-c7f6922f-3194-4d33-ac2e-7304d526eb94 None None] Registered counter for database keystone May 19 06:14:55.732455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG oslo_db.sqlalchemy.engines [None req-c7f6922f-3194-4d33-ac2e-7304d526eb94 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80281) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 06:14:55.740030 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writer thread running {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 06:14:55.945101 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG passlib.handlers.bcrypt [None req-c7f6922f-3194-4d33-ac2e-7304d526eb94 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80281) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 06:14:55.945566 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG passlib.handlers.bcrypt [None req-c7f6922f-3194-4d33-ac2e-7304d526eb94 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80281) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 06:14:56.003342 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c7f6922f-3194-4d33-ac2e-7304d526eb94 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:14:56.005030 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c7f6922f-3194-4d33-ac2e-7304d526eb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:14:56.147985 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1/3] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:14:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 516 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 19 06:14:56.202927 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3595d-456a-44a5-ae02-b2cba217ca33 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:14:56.274665 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: INFO dbcounter [None req-2be3595d-456a-44a5-ae02-b2cba217ca33 None None] Registered counter for database keystone May 19 06:14:56.304116 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG oslo_db.sqlalchemy.engines [None req-2be3595d-456a-44a5-ae02-b2cba217ca33 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80280) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 06:14:56.311528 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writer thread running {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 06:14:56.440479 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3595d-456a-44a5-ae02-b2cba217ca33 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:14:56.445469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caddd1a-9e1b-4936-ba60-bf6af43a69a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:14:56.446281 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caddd1a-9e1b-4936-ba60-bf6af43a69a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:14:56.446662 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caddd1a-9e1b-4936-ba60-bf6af43a69a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:14:56.447037 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caddd1a-9e1b-4936-ba60-bf6af43a69a1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:14:56.457312 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caddd1a-9e1b-4936-ba60-bf6af43a69a1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:14:56.509743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9caddd1a-9e1b-4936-ba60-bf6af43a69a1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 06:14:56.510206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caddd1a-9e1b-4936-ba60-bf6af43a69a1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:14:56.522033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 3/4] 10.222.0.65 () {58 vars in 1324 bytes} [Thu May 19 06:14:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 06:14:56.528471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6922f-3194-4d33-ac2e-7304d526eb94 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:14:56.544686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6922f-3194-4d33-ac2e-7304d526eb94 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:14:56.549877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87fc81-eeea-4f14-a0cf-50263ee93c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:14:56.551099 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87fc81-eeea-4f14-a0cf-50263ee93c70 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:14:56.551440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87fc81-eeea-4f14-a0cf-50263ee93c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:14:56.551770 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87fc81-eeea-4f14-a0cf-50263ee93c70 admin admin] PATH_INFO: `/v3/projects/0443648214584699a9ada0deda814964` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:14:56.556391 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87fc81-eeea-4f14-a0cf-50263ee93c70 admin admin] RBAC: Authorizing `identity:get_project(project_id=0443648214584699a9ada0deda814964)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:14:56.608898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ef87fc81-eeea-4f14-a0cf-50263ee93c70 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 06:14:56.609234 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87fc81-eeea-4f14-a0cf-50263ee93c70 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:14:56.614678 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 2/5] 10.222.0.65 () {56 vars in 1207 bytes} [Thu May 19 06:14:56 2022] GET /identity/v3/projects/0443648214584699a9ada0deda814964 => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:14:58.935704 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30eb5c-9947-457f-ab15-8de6fcb1d3fb None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:14:58.936293 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30eb5c-9947-457f-ab15-8de6fcb1d3fb None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:14:58.936293 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30eb5c-9947-457f-ab15-8de6fcb1d3fb None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:14:58.941795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 4/6] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:14:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:14:58.947924 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94b288-a8a9-4e85-ae1c-fbf5e9debf75 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:14:58.948446 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94b288-a8a9-4e85-ae1c-fbf5e9debf75 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:14:58.948573 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94b288-a8a9-4e85-ae1c-fbf5e9debf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:14:58.968240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4c94b288-a8a9-4e85-ae1c-fbf5e9debf75 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:14:58.969929 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4c94b288-a8a9-4e85-ae1c-fbf5e9debf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:14:58.978277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 3/7] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:14:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 06:14:59.002728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30eb5c-9947-457f-ab15-8de6fcb1d3fb None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:14:59.015278 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30eb5c-9947-457f-ab15-8de6fcb1d3fb None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:14:59.020300 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99adf4e2-a9aa-45c6-928a-6ca791a8eb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:14:59.020964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adf4e2-a9aa-45c6-928a-6ca791a8eb4e admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:14:59.021258 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adf4e2-a9aa-45c6-928a-6ca791a8eb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:14:59.021576 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adf4e2-a9aa-45c6-928a-6ca791a8eb4e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:14:59.030886 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99adf4e2-a9aa-45c6-928a-6ca791a8eb4e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:14:59.033803 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99adf4e2-a9aa-45c6-928a-6ca791a8eb4e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:14:59.043947 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 5/8] 10.222.0.65 () {58 vars in 1324 bytes} [Thu May 19 06:14:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 06:14:59.054443 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94b288-a8a9-4e85-ae1c-fbf5e9debf75 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:14:59.071821 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94b288-a8a9-4e85-ae1c-fbf5e9debf75 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:14:59.079948 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bf1a2-7bc4-43a8-9675-758296bc7cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:14:59.081609 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bf1a2-7bc4-43a8-9675-758296bc7cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:14:59.082100 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bf1a2-7bc4-43a8-9675-758296bc7cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:14:59.082613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bf1a2-7bc4-43a8-9675-758296bc7cc4 admin admin] PATH_INFO: `/v3/users/5a11fde32c844e228722284109f82f80` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:14:59.085500 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bf1a2-7bc4-43a8-9675-758296bc7cc4 admin admin] RBAC: Authorizing `identity:get_user(user_id=5a11fde32c844e228722284109f82f80)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:14:59.089042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bf1a2-7bc4-43a8-9675-758296bc7cc4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:14:59.092240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 4/9] 10.222.0.65 () {56 vars in 1198 bytes} [Thu May 19 06:14:59 2022] GET /identity/v3/users/5a11fde32c844e228722284109f82f80 => generated 247 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:01.507622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acaa4f-bc47-41bc-88cf-99a1665968de None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:01.508204 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acaa4f-bc47-41bc-88cf-99a1665968de None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:01.508550 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acaa4f-bc47-41bc-88cf-99a1665968de None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:01.510959 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 6/10] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:01.520671 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7095c-13f3-4108-af90-674b356c9924 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:01.521217 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7095c-13f3-4108-af90-674b356c9924 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:01.521734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7095c-13f3-4108-af90-674b356c9924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:01.532545 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85532ad-5fd6-4ec6-8190-488c24b9e4aa None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:01.532832 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85532ad-5fd6-4ec6-8190-488c24b9e4aa None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:01.533121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85532ad-5fd6-4ec6-8190-488c24b9e4aa None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:01.534593 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 7/11] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:15:01.540356 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536eef3c-4b67-4ff7-abfc-c1861b4d9eff None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:01.540646 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536eef3c-4b67-4ff7-abfc-c1861b4d9eff None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:01.540934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536eef3c-4b67-4ff7-abfc-c1861b4d9eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:01.547957 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-28a7095c-13f3-4108-af90-674b356c9924 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:01.550015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-28a7095c-13f3-4108-af90-674b356c9924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:01.583116 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG passlib.handlers.bcrypt [None req-536eef3c-4b67-4ff7-abfc-c1861b4d9eff None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80280) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 06:15:01.583597 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG passlib.handlers.bcrypt [None req-536eef3c-4b67-4ff7-abfc-c1861b4d9eff None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80280) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 06:15:01.631012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-536eef3c-4b67-4ff7-abfc-c1861b4d9eff None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:01.632887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-536eef3c-4b67-4ff7-abfc-c1861b4d9eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:01.675591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 5/12] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:01.706438 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7095c-13f3-4108-af90-674b356c9924 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:01.732086 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:01.769750 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 8/13] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:15:01.808930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536eef3c-4b67-4ff7-abfc-c1861b4d9eff None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:01.865855 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7095c-13f3-4108-af90-674b356c9924 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:01.911952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1add66-91ff-43bc-bc0f-3a7ff75ef849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:01.912915 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1add66-91ff-43bc-bc0f-3a7ff75ef849 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:01.913314 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1add66-91ff-43bc-bc0f-3a7ff75ef849 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:01.913698 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1add66-91ff-43bc-bc0f-3a7ff75ef849 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:01.928918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1add66-91ff-43bc-bc0f-3a7ff75ef849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:01.940630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-ee1add66-91ff-43bc-bc0f-3a7ff75ef849 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:01.940630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:01.940630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:01.940630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:01.940630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:01.940630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:01.940630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:01.940630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:01.940630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:01.940630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:01.940630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:01.940630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:01.940630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:01.940630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:01.940630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:01.940630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:01.942545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:01.942545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:01.942545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:01.942545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:01.942545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:01.942545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:01.942545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:01.942545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:01.942545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:01.942545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:01.942545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:01.942545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:15:01.942545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:01.942545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:01.942545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:01.942545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:01.943980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:01.943980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:01.943980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:01.943980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:01.943980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:01.943980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:01.943980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:01.943980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:01.943980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:01.943980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:01.943980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:01.943980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:15:01.943980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 6/14] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:15:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:15:01.948182 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1add66-91ff-43bc-bc0f-3a7ff75ef849 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:02.049314 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:02.050611 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536eef3c-4b67-4ff7-abfc-c1861b4d9eff None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:02.053782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:02.073440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1add66-91ff-43bc-bc0f-3a7ff75ef849 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:02.113132 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f458558-5c55-40c7-a231-532f8e56c9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:02.114106 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f458558-5c55-40c7-a231-532f8e56c9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:02.114396 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f458558-5c55-40c7-a231-532f8e56c9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:02.114677 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f458558-5c55-40c7-a231-532f8e56c9b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:02.123008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8babf4-54aa-4183-88f8-209753ffafbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:02.124093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8babf4-54aa-4183-88f8-209753ffafbb None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:02.124618 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8babf4-54aa-4183-88f8-209753ffafbb None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:02.125152 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8babf4-54aa-4183-88f8-209753ffafbb None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:02.126690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8babf4-54aa-4183-88f8-209753ffafbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:02.127267 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8babf4-54aa-4183-88f8-209753ffafbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:02.130023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8babf4-54aa-4183-88f8-209753ffafbb None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:02.157720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 7/15] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:02.160499 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8babf4-54aa-4183-88f8-209753ffafbb None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:02.192093 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f458558-5c55-40c7-a231-532f8e56c9b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:02.193697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f458558-5c55-40c7-a231-532f8e56c9b4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:02.300681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 9/16] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 06:15:02.309000 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f458558-5c55-40c7-a231-532f8e56c9b4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:02.312323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:02.327804 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8babf4-54aa-4183-88f8-209753ffafbb None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:02.372647 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5e4df-17eb-4bdc-a8a3-926c44ef7d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:02.373782 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5e4df-17eb-4bdc-a8a3-926c44ef7d88 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:02.374134 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5e4df-17eb-4bdc-a8a3-926c44ef7d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:02.374503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5e4df-17eb-4bdc-a8a3-926c44ef7d88 None admin] PATH_INFO: `/v3/users/5a11fde32c844e228722284109f82f80` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:02.376531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5e4df-17eb-4bdc-a8a3-926c44ef7d88 None admin] RBAC: Authorizing `identity:get_user(user_id=5a11fde32c844e228722284109f82f80)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:02.378445 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5e4df-17eb-4bdc-a8a3-926c44ef7d88 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:02.380341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 8/17] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:02 2022] GET /identity/v3/users/5a11fde32c844e228722284109f82f80 => generated 247 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:02.386241 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5e4df-17eb-4bdc-a8a3-926c44ef7d88 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:02.404198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:02.437710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f458558-5c55-40c7-a231-532f8e56c9b4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:02.480628 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650cc932-8f5b-4924-9265-d0afe669c7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:02.481618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cc932-8f5b-4924-9265-d0afe669c7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:02.481977 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cc932-8f5b-4924-9265-d0afe669c7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:02.482342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cc932-8f5b-4924-9265-d0afe669c7c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:02.511770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650cc932-8f5b-4924-9265-d0afe669c7c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:02.514442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650cc932-8f5b-4924-9265-d0afe669c7c1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:02.519328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5e4df-17eb-4bdc-a8a3-926c44ef7d88 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:02.551549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-650cc932-8f5b-4924-9265-d0afe669c7c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:15:02.551549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:02.551549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:02.551549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:02.551549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:02.551549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:02.551549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:02.551549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:02.551549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:02.551549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:02.551549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:02.551549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:02.551549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:15:02.551549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:15:02.551549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:15:02.552519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:15:02.552519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:02.552519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:02.552519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:02.552519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:02.552519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:02.552519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:02.552519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:02.552519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:15:02.552519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:02.552519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:02.552519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:02.552519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:02.552519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:02.552519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:02.552519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:02.552519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:02.553379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:15:02.553379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:15:02.553379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:15:02.553379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:15:02.553663 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 10/18] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:15:02.563589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650cc932-8f5b-4924-9265-d0afe669c7c1 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:02.663125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:02.727903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6baa0a-0e00-40cd-8ff3-c07533a39183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:02.728945 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6baa0a-0e00-40cd-8ff3-c07533a39183 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:02.729352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6baa0a-0e00-40cd-8ff3-c07533a39183 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:02.729714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6baa0a-0e00-40cd-8ff3-c07533a39183 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:02.730744 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650cc932-8f5b-4924-9265-d0afe669c7c1 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:02.731113 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6baa0a-0e00-40cd-8ff3-c07533a39183 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:02.732806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6baa0a-0e00-40cd-8ff3-c07533a39183 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:02.734823 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 9/19] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:15:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:02.741089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6baa0a-0e00-40cd-8ff3-c07533a39183 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:02.770916 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af4f2a-94f2-4f15-82ba-da3ebcd17df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:02.771622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af4f2a-94f2-4f15-82ba-da3ebcd17df1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:02.771879 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af4f2a-94f2-4f15-82ba-da3ebcd17df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:02.772157 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af4f2a-94f2-4f15-82ba-da3ebcd17df1 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:02.772899 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af4f2a-94f2-4f15-82ba-da3ebcd17df1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:02.773204 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af4f2a-94f2-4f15-82ba-da3ebcd17df1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:02.774651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af4f2a-94f2-4f15-82ba-da3ebcd17df1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:02.805645 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 11/20] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:02.842711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:02.868767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6baa0a-0e00-40cd-8ff3-c07533a39183 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:02.912993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15486a-201e-4962-9add-33603b9694ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:02.913784 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15486a-201e-4962-9add-33603b9694ea None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:02.914004 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15486a-201e-4962-9add-33603b9694ea None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:02.914290 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15486a-201e-4962-9add-33603b9694ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:02.915154 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15486a-201e-4962-9add-33603b9694ea None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.domain.id=default, user.id=5a11fde32c844e228722284109f82f80) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:02.915474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15486a-201e-4962-9add-33603b9694ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:02.916985 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15486a-201e-4962-9add-33603b9694ea None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:02.925810 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 10/21] 10.222.0.65 () {56 vars in 1322 bytes} [Thu May 19 06:15:02 2022] GET /identity/v3/role_assignments?user.id=5a11fde32c844e228722284109f82f80&scope.domain.id=default&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 233 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:06.054188 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602db5b-4cde-43d1-8e7a-a86f3f320c35 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:06.054711 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602db5b-4cde-43d1-8e7a-a86f3f320c35 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:06.054812 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602db5b-4cde-43d1-8e7a-a86f3f320c35 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:06.056403 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 12/22] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:06.065930 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39baae-efe0-4fac-9d7f-0a2a14b4b329 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:06.067421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39baae-efe0-4fac-9d7f-0a2a14b4b329 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:06.068216 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39baae-efe0-4fac-9d7f-0a2a14b4b329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:06.110770 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9e39baae-efe0-4fac-9d7f-0a2a14b4b329 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:06.113406 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9e39baae-efe0-4fac-9d7f-0a2a14b4b329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:06.298153 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 11/23] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:06.338882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602db5b-4cde-43d1-8e7a-a86f3f320c35 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:06.427376 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877f1ce-1bfe-4847-b8af-1c23e22e8bca None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:06.427688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877f1ce-1bfe-4847-b8af-1c23e22e8bca None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:06.427946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877f1ce-1bfe-4847-b8af-1c23e22e8bca None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:06.429343 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 12/24] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:15:06.443156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fe527-db66-47cd-80aa-65cd28771515 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:06.443454 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fe527-db66-47cd-80aa-65cd28771515 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:06.443875 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fe527-db66-47cd-80aa-65cd28771515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:06.474384 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1a6fe527-db66-47cd-80aa-65cd28771515 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:06.475617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1a6fe527-db66-47cd-80aa-65cd28771515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:06.504807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602db5b-4cde-43d1-8e7a-a86f3f320c35 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:06.555069 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:06.613978 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615a2e7-21e5-40a3-a3f3-4a0f6b4fc298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:06.616305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615a2e7-21e5-40a3-a3f3-4a0f6b4fc298 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:06.617127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615a2e7-21e5-40a3-a3f3-4a0f6b4fc298 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:06.617924 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615a2e7-21e5-40a3-a3f3-4a0f6b4fc298 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:06.625440 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615a2e7-21e5-40a3-a3f3-4a0f6b4fc298 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:06.629091 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615a2e7-21e5-40a3-a3f3-4a0f6b4fc298 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:06.659277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 13/25] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:15:06.766437 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fe527-db66-47cd-80aa-65cd28771515 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:06.773660 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:06.924002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 13/26] 10.222.0.65 () {60 vars in 1131 bytes} [Thu May 19 06:15:06 2022] POST /identity/v3/roles => generated 219 bytes in 585 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:15:06.960802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712e32e-dfe2-4403-97e1-763d59f68948 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:06.961790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712e32e-dfe2-4403-97e1-763d59f68948 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:06.962601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712e32e-dfe2-4403-97e1-763d59f68948 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:06.966201 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 14/27] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:06.974766 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28547d58-cf0e-4b32-a238-89059b77ddf9 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:06.975716 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28547d58-cf0e-4b32-a238-89059b77ddf9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:06.976294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28547d58-cf0e-4b32-a238-89059b77ddf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:07.005618 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fe527-db66-47cd-80aa-65cd28771515 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:07.021442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-28547d58-cf0e-4b32-a238-89059b77ddf9 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:07.023611 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-28547d58-cf0e-4b32-a238-89059b77ddf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:07.069209 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b3865-9bd6-48ea-b5e9-30210030d871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:07.070865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b3865-9bd6-48ea-b5e9-30210030d871 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:07.071408 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b3865-9bd6-48ea-b5e9-30210030d871 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:07.071886 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b3865-9bd6-48ea-b5e9-30210030d871 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:07.085204 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b3865-9bd6-48ea-b5e9-30210030d871 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:07.087029 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b3865-9bd6-48ea-b5e9-30210030d871 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:07.108987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-b99b3865-9bd6-48ea-b5e9-30210030d871 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:15:07.108987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:07.108987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:07.108987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:07.108987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:07.108987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:07.108987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:07.108987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:07.108987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:07.108987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:07.108987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:07.108987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:07.108987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:15:07.108987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:15:07.108987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:15:07.110724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:15:07.110724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:07.110724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:07.110724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:07.110724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:07.110724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:07.110724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:07.110724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:07.110724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:15:07.110724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:07.110724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:07.110724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:07.110724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:07.110724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:07.110724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:07.110724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:07.110724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:07.111605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:15:07.111605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:15:07.111605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:15:07.111605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:15:07.113483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 14/28] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:15:07.137039 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b3865-9bd6-48ea-b5e9-30210030d871 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:07.206925 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 15/29] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:07.237158 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28547d58-cf0e-4b32-a238-89059b77ddf9 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:07.333317 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:07.394814 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b3865-9bd6-48ea-b5e9-30210030d871 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:07.437823 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b4fd2-4bbd-4506-834c-faa0a3d1f6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:07.438518 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b4fd2-4bbd-4506-834c-faa0a3d1f6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:07.438865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b4fd2-4bbd-4506-834c-faa0a3d1f6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:07.439079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b4fd2-4bbd-4506-834c-faa0a3d1f6f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:07.439889 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b4fd2-4bbd-4506-834c-faa0a3d1f6f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:07.440193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b4fd2-4bbd-4506-834c-faa0a3d1f6f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:07.441741 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b4fd2-4bbd-4506-834c-faa0a3d1f6f0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:07.462412 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 15/30] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:07.463822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e5bef-7e12-472f-8ec7-dcec61e1372d None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:07.464061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e5bef-7e12-472f-8ec7-dcec61e1372d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:07.464297 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e5bef-7e12-472f-8ec7-dcec61e1372d None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:07.465711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 16/31] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:07.473451 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28547d58-cf0e-4b32-a238-89059b77ddf9 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:07.477106 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e5bef-7e12-472f-8ec7-dcec61e1372d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:07.491251 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:07.557999 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e461b-fb52-4c13-952a-3b2688391bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:07.559438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e461b-fb52-4c13-952a-3b2688391bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:07.559936 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e461b-fb52-4c13-952a-3b2688391bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:07.560442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e461b-fb52-4c13-952a-3b2688391bd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:07.588548 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e461b-fb52-4c13-952a-3b2688391bd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:07.611029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f57e461b-fb52-4c13-952a-3b2688391bd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:07.611029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:07.611029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:07.611029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:07.611029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:07.611029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:07.611029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:07.611029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:07.611029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:07.611029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:07.611029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:07.611029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:07.611029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:07.611029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:07.611029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:07.611029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:07.612762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:07.612762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:07.612762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:07.612762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:07.612762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:07.612762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:07.612762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:07.612762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:07.612762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:07.612762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:07.612762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:07.612762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:15:07.612762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:07.612762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:07.612762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:07.612762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:07.614348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:07.614348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:07.614348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:07.614348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:07.614348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:07.614348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:07.614348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:07.614348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:07.614348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:07.614348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:07.614348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:07.614348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:15:07.614348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 16/32] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:15:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:07.616511 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f3257-30fb-425a-9f09-a0df175bb1e9 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:07.620187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f3257-30fb-425a-9f09-a0df175bb1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:07.620187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f3257-30fb-425a-9f09-a0df175bb1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:07.649249 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e5bef-7e12-472f-8ec7-dcec61e1372d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:07.658115 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-934f3257-30fb-425a-9f09-a0df175bb1e9 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:07.660260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-934f3257-30fb-425a-9f09-a0df175bb1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:07.679526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:07.711900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba00a4-8674-4808-8bdc-d3dca3f4eb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:07.713132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba00a4-8674-4808-8bdc-d3dca3f4eb83 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:07.713627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba00a4-8674-4808-8bdc-d3dca3f4eb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:07.714142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba00a4-8674-4808-8bdc-d3dca3f4eb83 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:07.715511 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ba00a4-8674-4808-8bdc-d3dca3f4eb83 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:07.717882 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ba00a4-8674-4808-8bdc-d3dca3f4eb83 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:07.778469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 17/33] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:07.779712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f3257-30fb-425a-9f09-a0df175bb1e9 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:07.901002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:07.909956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f3257-30fb-425a-9f09-a0df175bb1e9 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:07.950713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74518a43-55d6-46fa-aaa0-983a1eaa19b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:07.951506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74518a43-55d6-46fa-aaa0-983a1eaa19b5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:07.951895 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74518a43-55d6-46fa-aaa0-983a1eaa19b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:07.952265 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74518a43-55d6-46fa-aaa0-983a1eaa19b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:07.953723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74518a43-55d6-46fa-aaa0-983a1eaa19b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:07.954132 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74518a43-55d6-46fa-aaa0-983a1eaa19b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:07.954964 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 17/34] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 06:15:07 2022] POST /identity/v3/projects => generated 301 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:15:07.955929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74518a43-55d6-46fa-aaa0-983a1eaa19b5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:07.957720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0d7ca-28d1-4a8b-812d-893b9160fe35 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:07.958230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0d7ca-28d1-4a8b-812d-893b9160fe35 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:07.958698 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0d7ca-28d1-4a8b-812d-893b9160fe35 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:07.961093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 18/35] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:07.963357 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9903e44-4b47-4dda-813a-e3a974fe241c None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:07.963748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9903e44-4b47-4dda-813a-e3a974fe241c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:07.964348 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9903e44-4b47-4dda-813a-e3a974fe241c None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:07.966765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 19/36] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:07.967851 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9903e44-4b47-4dda-813a-e3a974fe241c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:07.985826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 18/37] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:07.989669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4255-a4c0-41f5-9ecb-88c3bc9e0d94 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:07.990583 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4255-a4c0-41f5-9ecb-88c3bc9e0d94 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:07.992163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4255-a4c0-41f5-9ecb-88c3bc9e0d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:08.016208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-288f4255-a4c0-41f5-9ecb-88c3bc9e0d94 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:08.018262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-288f4255-a4c0-41f5-9ecb-88c3bc9e0d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:08.133254 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 19/38] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:08.135496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762b6e1-64e8-4602-85b8-272b0a50dbe7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:08.135982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762b6e1-64e8-4602-85b8-272b0a50dbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:08.136456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762b6e1-64e8-4602-85b8-272b0a50dbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:08.164815 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9903e44-4b47-4dda-813a-e3a974fe241c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:08.168397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3762b6e1-64e8-4602-85b8-272b0a50dbe7 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:08.170550 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3762b6e1-64e8-4602-85b8-272b0a50dbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:08.220206 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c58c3-3db3-4868-8234-8ff61375c74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:08.221252 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c58c3-3db3-4868-8234-8ff61375c74b None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:08.221785 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c58c3-3db3-4868-8234-8ff61375c74b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:08.222066 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c58c3-3db3-4868-8234-8ff61375c74b None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:08.222885 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c58c3-3db3-4868-8234-8ff61375c74b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:08.224541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c58c3-3db3-4868-8234-8ff61375c74b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:08.248712 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 20/39] 10.222.0.65 () {60 vars in 1131 bytes} [Thu May 19 06:15:07 2022] POST /identity/v3/roles => generated 225 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:15:08.253343 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c58c3-3db3-4868-8234-8ff61375c74b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:08.316831 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:08.322875 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 20/40] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:08.322875 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762b6e1-64e8-4602-85b8-272b0a50dbe7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:08.375928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:08.436745 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c58c3-3db3-4868-8234-8ff61375c74b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:08.484721 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665aece-a3b0-4324-97a0-0a9ccef6f3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:08.486049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665aece-a3b0-4324-97a0-0a9ccef6f3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:08.486322 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665aece-a3b0-4324-97a0-0a9ccef6f3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:08.486589 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665aece-a3b0-4324-97a0-0a9ccef6f3d0 None admin] PATH_INFO: `/v3/users/5a11fde32c844e228722284109f82f80` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:08.487973 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665aece-a3b0-4324-97a0-0a9ccef6f3d0 None admin] RBAC: Authorizing `identity:get_user(user_id=5a11fde32c844e228722284109f82f80)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:08.489555 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665aece-a3b0-4324-97a0-0a9ccef6f3d0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:08.498849 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 21/41] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:08 2022] GET /identity/v3/users/5a11fde32c844e228722284109f82f80 => generated 247 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:08.498849 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665aece-a3b0-4324-97a0-0a9ccef6f3d0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:08.542124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762b6e1-64e8-4602-85b8-272b0a50dbe7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:08.605108 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90090d-4764-4d6e-a1b5-7076c575cce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:08.606288 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90090d-4764-4d6e-a1b5-7076c575cce9 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:08.606736 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90090d-4764-4d6e-a1b5-7076c575cce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:08.607173 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90090d-4764-4d6e-a1b5-7076c575cce9 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:08.608379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90090d-4764-4d6e-a1b5-7076c575cce9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:08.610695 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90090d-4764-4d6e-a1b5-7076c575cce9 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:08.628627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 21/42] 10.222.0.65 () {60 vars in 1131 bytes} [Thu May 19 06:15:08 2022] POST /identity/v3/roles => generated 223 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:15:08.629959 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90090d-4764-4d6e-a1b5-7076c575cce9 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:08.652991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:08.663547 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665aece-a3b0-4324-97a0-0a9ccef6f3d0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:08.719376 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f5c9b-ec47-4e8c-abb9-fd3543e4c718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:08.721173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f5c9b-ec47-4e8c-abb9-fd3543e4c718 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:08.721678 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f5c9b-ec47-4e8c-abb9-fd3543e4c718 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:08.722121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f5c9b-ec47-4e8c-abb9-fd3543e4c718 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:08.723842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f5c9b-ec47-4e8c-abb9-fd3543e4c718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:08.725546 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f5c9b-ec47-4e8c-abb9-fd3543e4c718 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:08.727411 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 22/43] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:15:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:08.738222 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f5c9b-ec47-4e8c-abb9-fd3543e4c718 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:08.769756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:08.814475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90090d-4764-4d6e-a1b5-7076c575cce9 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:08.860806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c830ad-734f-4e99-ac85-be2b803fbcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:08.862093 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c830ad-734f-4e99-ac85-be2b803fbcbd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:08.862379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c830ad-734f-4e99-ac85-be2b803fbcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:08.862659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c830ad-734f-4e99-ac85-be2b803fbcbd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:08.864342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c830ad-734f-4e99-ac85-be2b803fbcbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:08.866555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c830ad-734f-4e99-ac85-be2b803fbcbd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:08.868798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 22/44] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:15:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:08.874587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c830ad-734f-4e99-ac85-be2b803fbcbd None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:08.886874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f5c9b-ec47-4e8c-abb9-fd3543e4c718 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:08.944172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c78d1-8758-429d-9746-5e1f631bde34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:08.944172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c78d1-8758-429d-9746-5e1f631bde34 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:08.944594 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c78d1-8758-429d-9746-5e1f631bde34 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:08.944950 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c78d1-8758-429d-9746-5e1f631bde34 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:08.946493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c78d1-8758-429d-9746-5e1f631bde34 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:08.947087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c78d1-8758-429d-9746-5e1f631bde34 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:08.993876 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 23/45] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 06:15:08 2022] POST /identity/v3/projects => generated 312 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:15:09.046828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c830ad-734f-4e99-ac85-be2b803fbcbd None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:09.090555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00c149-3233-4ff9-9a2e-cf9ef69b7c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:09.091474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00c149-3233-4ff9-9a2e-cf9ef69b7c8d None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:09.091763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00c149-3233-4ff9-9a2e-cf9ef69b7c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:09.092059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00c149-3233-4ff9-9a2e-cf9ef69b7c8d None admin] PATH_INFO: `/v3/domains/default/users/5a11fde32c844e228722284109f82f80/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:09.097867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00c149-3233-4ff9-9a2e-cf9ef69b7c8d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5a11fde32c844e228722284109f82f80, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:09.100496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00c149-3233-4ff9-9a2e-cf9ef69b7c8d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:09.122957 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 23/46] 10.222.0.65 () {58 vars in 1361 bytes} [Thu May 19 06:15:08 2022] PUT /identity/v3/domains/default/users/5a11fde32c844e228722284109f82f80/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:15:11.394644 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343236e-b79a-489e-a089-c0ad41e95971 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:11.395410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343236e-b79a-489e-a089-c0ad41e95971 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:11.395410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343236e-b79a-489e-a089-c0ad41e95971 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:11.396786 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 24/47] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:11.405938 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d376f-ed74-4702-a640-3d9499573236 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:11.406495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d376f-ed74-4702-a640-3d9499573236 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:11.406994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d376f-ed74-4702-a640-3d9499573236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:11.427058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:11.436569 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-bb9d376f-ed74-4702-a640-3d9499573236 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:11.440357 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bb9d376f-ed74-4702-a640-3d9499573236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:11.542253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d050f-f5f6-46be-ba64-fe7f74964db9 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:11.542641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d050f-f5f6-46be-ba64-fe7f74964db9 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:11.543011 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d050f-f5f6-46be-ba64-fe7f74964db9 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:11.544681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 25/48] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:15:11.551552 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5f538-bcea-407e-81b0-ac96a527e49a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:11.551948 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5f538-bcea-407e-81b0-ac96a527e49a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:11.552287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5f538-bcea-407e-81b0-ac96a527e49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:11.559857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 24/49] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:11.571398 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f4a5f538-bcea-407e-81b0-ac96a527e49a None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:11.573070 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f4a5f538-bcea-407e-81b0-ac96a527e49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:11.585960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d376f-ed74-4702-a640-3d9499573236 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:11.586185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:11.668622 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 26/50] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:15:11.711725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5f538-bcea-407e-81b0-ac96a527e49a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:11.738587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d376f-ed74-4702-a640-3d9499573236 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:11.813944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82813ec4-5196-4795-b118-be9d9f94ddf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:11.815362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82813ec4-5196-4795-b118-be9d9f94ddf0 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:11.815866 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82813ec4-5196-4795-b118-be9d9f94ddf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:11.816385 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82813ec4-5196-4795-b118-be9d9f94ddf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:11.818571 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82813ec4-5196-4795-b118-be9d9f94ddf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:11.821479 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82813ec4-5196-4795-b118-be9d9f94ddf0 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:11.824812 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 25/51] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:15:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:11.831590 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82813ec4-5196-4795-b118-be9d9f94ddf0 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:11.837932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:11.848461 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5f538-bcea-407e-81b0-ac96a527e49a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:11.889756 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8986675-b4d2-41bc-b252-586c923bfc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:11.891233 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8986675-b4d2-41bc-b252-586c923bfc91 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:11.891523 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8986675-b4d2-41bc-b252-586c923bfc91 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:11.891809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8986675-b4d2-41bc-b252-586c923bfc91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:11.903406 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8986675-b4d2-41bc-b252-586c923bfc91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:11.912545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-e8986675-b4d2-41bc-b252-586c923bfc91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:11.912545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:11.912545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:11.912545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:11.912545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:11.912545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:11.912545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:11.912545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:11.912545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:11.912545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:11.912545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:11.912545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:11.912545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:11.912545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:11.912545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:11.912545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:11.914287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:11.914287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:11.914287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:11.914287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:11.914287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:11.914287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:11.914287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:11.914287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:11.914287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:11.914287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:11.914287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:11.914287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:15:11.914287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:11.914287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:11.914287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:11.914287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:11.915328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:11.915328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:11.915328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:11.915328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:11.915328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:11.915328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:11.915328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:11.915328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:11.915328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:11.915328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:11.915328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:11.915328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:15:11.915328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 27/52] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:15:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:15:11.924067 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8986675-b4d2-41bc-b252-586c923bfc91 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:11.934344 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:11.978279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82813ec4-5196-4795-b118-be9d9f94ddf0 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:12.035049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b56511-bad1-4442-82d7-23e8ca64adb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:12.035849 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b56511-bad1-4442-82d7-23e8ca64adb5 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:12.036200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b56511-bad1-4442-82d7-23e8ca64adb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:12.036545 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b56511-bad1-4442-82d7-23e8ca64adb5 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:12.037421 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b56511-bad1-4442-82d7-23e8ca64adb5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:12.038983 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b56511-bad1-4442-82d7-23e8ca64adb5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:12.060647 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8986675-b4d2-41bc-b252-586c923bfc91 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:12.071526 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 26/53] 10.222.0.65 () {60 vars in 1140 bytes} [Thu May 19 06:15:11 2022] POST /identity/v3/projects => generated 298 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:15:12.129333 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dffe11-ee63-4d34-83b8-2fd4da2ada4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:12.130430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dffe11-ee63-4d34-83b8-2fd4da2ada4d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:12.130955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dffe11-ee63-4d34-83b8-2fd4da2ada4d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:12.131464 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dffe11-ee63-4d34-83b8-2fd4da2ada4d None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:12.132967 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dffe11-ee63-4d34-83b8-2fd4da2ada4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:12.133573 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dffe11-ee63-4d34-83b8-2fd4da2ada4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:12.136318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dffe11-ee63-4d34-83b8-2fd4da2ada4d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:12.159911 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 28/54] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:12.161494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b56511-bad1-4442-82d7-23e8ca64adb5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:12.241160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:12.300200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b56511-bad1-4442-82d7-23e8ca64adb5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:12.341935 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f040c2-7f5a-4c4c-9dd1-d800cf6d7401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:12.342858 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f040c2-7f5a-4c4c-9dd1-d800cf6d7401 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:12.343148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f040c2-7f5a-4c4c-9dd1-d800cf6d7401 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:12.343462 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f040c2-7f5a-4c4c-9dd1-d800cf6d7401 None admin] PATH_INFO: `/v3/users/5a11fde32c844e228722284109f82f80` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:12.344832 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f040c2-7f5a-4c4c-9dd1-d800cf6d7401 None admin] RBAC: Authorizing `identity:get_user(user_id=5a11fde32c844e228722284109f82f80)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:12.346345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f040c2-7f5a-4c4c-9dd1-d800cf6d7401 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:12.348070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 27/55] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:12 2022] GET /identity/v3/users/5a11fde32c844e228722284109f82f80 => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:12.353736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dffe11-ee63-4d34-83b8-2fd4da2ada4d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:12.427750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:12.465188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dffe11-ee63-4d34-83b8-2fd4da2ada4d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:12.505712 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee933c-acbd-458e-b10d-f1b9b0a9b36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:12.506615 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee933c-acbd-458e-b10d-f1b9b0a9b36a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:12.506975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee933c-acbd-458e-b10d-f1b9b0a9b36a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:12.507320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee933c-acbd-458e-b10d-f1b9b0a9b36a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:12.508652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee933c-acbd-458e-b10d-f1b9b0a9b36a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:12.510262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee933c-acbd-458e-b10d-f1b9b0a9b36a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:12.512056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 29/56] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:15:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:12.519179 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f040c2-7f5a-4c4c-9dd1-d800cf6d7401 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:12.647574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f040c2-7f5a-4c4c-9dd1-d800cf6d7401 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:12.675852 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:12.689033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d96e2-60f0-4e27-9240-daead7f6875f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:12.689648 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d96e2-60f0-4e27-9240-daead7f6875f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:12.689944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d96e2-60f0-4e27-9240-daead7f6875f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:12.690229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d96e2-60f0-4e27-9240-daead7f6875f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:12.691096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d96e2-60f0-4e27-9240-daead7f6875f None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.domain.id=default, user.id=5a11fde32c844e228722284109f82f80) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:12.691414 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d96e2-60f0-4e27-9240-daead7f6875f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:12.692897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d96e2-60f0-4e27-9240-daead7f6875f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:12.702436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 28/57] 10.222.0.65 () {56 vars in 1322 bytes} [Thu May 19 06:15:12 2022] GET /identity/v3/role_assignments?user.id=5a11fde32c844e228722284109f82f80&scope.domain.id=default&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 530 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:14.516093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4ea17-5621-410d-bc0b-5ede58d1773b None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:14.516659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4ea17-5621-410d-bc0b-5ede58d1773b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:14.516955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4ea17-5621-410d-bc0b-5ede58d1773b None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:14.518517 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 30/58] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:14.525700 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45842b7c-f7af-4d2f-9f4e-809162b726ec None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:14.526248 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45842b7c-f7af-4d2f-9f4e-809162b726ec None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:14.526772 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45842b7c-f7af-4d2f-9f4e-809162b726ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:14.556782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-45842b7c-f7af-4d2f-9f4e-809162b726ec None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:14.558976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-45842b7c-f7af-4d2f-9f4e-809162b726ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:14.677379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 29/59] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:14.701839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4ea17-5621-410d-bc0b-5ede58d1773b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:14.808317 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4ea17-5621-410d-bc0b-5ede58d1773b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:14.847314 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcad5a-1cb4-43c4-8514-67c51867ca65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:14.848172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcad5a-1cb4-43c4-8514-67c51867ca65 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:14.848512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcad5a-1cb4-43c4-8514-67c51867ca65 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:14.848843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcad5a-1cb4-43c4-8514-67c51867ca65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:14.850162 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcad5a-1cb4-43c4-8514-67c51867ca65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:14.851691 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcad5a-1cb4-43c4-8514-67c51867ca65 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:14.853418 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 31/60] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:15:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:14.859075 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45842b7c-f7af-4d2f-9f4e-809162b726ec None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:14.970285 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45842b7c-f7af-4d2f-9f4e-809162b726ec None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:14.989480 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:15.009514 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c660a4-1ade-4e9e-b6e0-94eb7a4a51a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:15.010206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c660a4-1ade-4e9e-b6e0-94eb7a4a51a7 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:15.010493 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c660a4-1ade-4e9e-b6e0-94eb7a4a51a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:15.010778 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c660a4-1ade-4e9e-b6e0-94eb7a4a51a7 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:15.011501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c660a4-1ade-4e9e-b6e0-94eb7a4a51a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:15.012999 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c660a4-1ade-4e9e-b6e0-94eb7a4a51a7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:15.042337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 30/61] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:15:14 2022] POST /identity/v3/users => generated 275 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:15:18.991477 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643065e8-b6ff-4929-9e2d-79f085798933 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:18.992239 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643065e8-b6ff-4929-9e2d-79f085798933 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:18.992239 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643065e8-b6ff-4929-9e2d-79f085798933 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:18.993698 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 32/62] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:19.006086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d3e05-3c1e-4545-ae77-d77055c0f324 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:19.006814 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d3e05-3c1e-4545-ae77-d77055c0f324 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:19.006814 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d3e05-3c1e-4545-ae77-d77055c0f324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:19.034499 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c78d3e05-3c1e-4545-ae77-d77055c0f324 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:19.035749 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c78d3e05-3c1e-4545-ae77-d77055c0f324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:19.138762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 31/63] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:19.209252 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643065e8-b6ff-4929-9e2d-79f085798933 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:19.213682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:19.376328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643065e8-b6ff-4929-9e2d-79f085798933 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:19.448827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f0d8c-2d19-457e-91dc-36ab428be1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:19.450252 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f0d8c-2d19-457e-91dc-36ab428be1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:19.450780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f0d8c-2d19-457e-91dc-36ab428be1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:19.451293 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f0d8c-2d19-457e-91dc-36ab428be1a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:19.487345 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f0d8c-2d19-457e-91dc-36ab428be1a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:19.491762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02d40c-771c-4954-99da-cc643b3f0495 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:19.492358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02d40c-771c-4954-99da-cc643b3f0495 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:19.492953 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02d40c-771c-4954-99da-cc643b3f0495 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:19.494627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 32/64] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:19.500383 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae2ad1-e678-4561-adfd-f5e38afcb8de None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:19.501014 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae2ad1-e678-4561-adfd-f5e38afcb8de None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:19.501569 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae2ad1-e678-4561-adfd-f5e38afcb8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:19.505329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-133f0d8c-2d19-457e-91dc-36ab428be1a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:19.505329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:19.505329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:19.505329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:19.505329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:19.505329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:19.505329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:19.505329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:19.505329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:19.505329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:19.505329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:19.505329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:19.505329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:19.505329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:19.505329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:19.505329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:19.506226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:19.506226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:19.506226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:19.506226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:19.506226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:19.506226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:19.506226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:19.506226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:19.506226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:19.506226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:19.506226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:19.506226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:15:19.506226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:19.506226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:19.506226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:19.506226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:19.507017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:19.507017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:19.507017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:19.507017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:19.507017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:19.507017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:19.507017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:19.507017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:19.507017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:19.507017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:19.507017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:19.507017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:15:19.507017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 33/65] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:15:19.516670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f0d8c-2d19-457e-91dc-36ab428be1a2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:19.542516 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-79ae2ad1-e678-4561-adfd-f5e38afcb8de None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:19.544479 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-79ae2ad1-e678-4561-adfd-f5e38afcb8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:19.674832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:19.745029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 33/66] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:15:19.769656 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae2ad1-e678-4561-adfd-f5e38afcb8de None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:19.785943 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f0d8c-2d19-457e-91dc-36ab428be1a2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:19.870274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52e9ba-2868-46cf-a7e1-773cede8e176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:19.871019 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52e9ba-2868-46cf-a7e1-773cede8e176 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:19.871296 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52e9ba-2868-46cf-a7e1-773cede8e176 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:19.871545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52e9ba-2868-46cf-a7e1-773cede8e176 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:19.872347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52e9ba-2868-46cf-a7e1-773cede8e176 None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:19.872663 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52e9ba-2868-46cf-a7e1-773cede8e176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:19.883806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52e9ba-2868-46cf-a7e1-773cede8e176 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:19.892618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:19.894341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 34/67] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:15:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:19.900506 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52e9ba-2868-46cf-a7e1-773cede8e176 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:20.051369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae2ad1-e678-4561-adfd-f5e38afcb8de None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:20.126228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3122646-1f67-4863-bdc3-83119df4d69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:20.128087 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3122646-1f67-4863-bdc3-83119df4d69e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:20.128611 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3122646-1f67-4863-bdc3-83119df4d69e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:20.129423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3122646-1f67-4863-bdc3-83119df4d69e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:20.133669 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52e9ba-2868-46cf-a7e1-773cede8e176 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:20.150246 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3122646-1f67-4863-bdc3-83119df4d69e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:20.164225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:20.168807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f3122646-1f67-4863-bdc3-83119df4d69e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:15:20.168807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:20.168807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:20.168807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:20.168807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:20.168807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:20.168807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:20.168807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:20.168807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:20.168807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:20.168807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:20.168807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:20.168807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:20.168807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:20.168807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:20.168807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:20.169774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:20.169774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:20.169774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:20.169774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:20.169774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:20.169774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:20.169774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:20.169774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:20.169774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:20.169774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:20.169774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:20.169774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:15:20.169774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:20.169774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:20.169774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:20.169774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:20.170951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:20.170951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:20.170951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:20.170951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:20.170951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:20.170951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:20.170951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:20.170951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:20.170951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:20.170951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:20.170951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:15:20.170951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:15:20.172850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 34/68] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:15:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:20.175103 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efd76f-cfad-41fc-bf8f-943eebe4dcd4 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:20.175842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efd76f-cfad-41fc-bf8f-943eebe4dcd4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:20.176546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efd76f-cfad-41fc-bf8f-943eebe4dcd4 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:20.178673 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58541bd1-4c44-4a98-b89a-73088baec6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:20.179574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 35/69] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:20.179834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58541bd1-4c44-4a98-b89a-73088baec6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:20.180226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58541bd1-4c44-4a98-b89a-73088baec6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:20.180612 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58541bd1-4c44-4a98-b89a-73088baec6b8 None admin] PATH_INFO: `/v3/users/b8be4e8c78444c25a8bd96f5851b92a1` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:20.182041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efd76f-cfad-41fc-bf8f-943eebe4dcd4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:20.198387 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58541bd1-4c44-4a98-b89a-73088baec6b8 None admin] RBAC: Authorizing `identity:get_user(user_id=b8be4e8c78444c25a8bd96f5851b92a1)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:20.200055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58541bd1-4c44-4a98-b89a-73088baec6b8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:20.209315 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 35/70] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:19 2022] GET /identity/v3/users/b8be4e8c78444c25a8bd96f5851b92a1 => generated 275 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:20.209315 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b820d3-10ef-415b-8811-1ab794446b12 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:20.209315 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b820d3-10ef-415b-8811-1ab794446b12 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:20.209315 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b820d3-10ef-415b-8811-1ab794446b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:20.224068 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-91b820d3-10ef-415b-8811-1ab794446b12 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:20.225840 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-91b820d3-10ef-415b-8811-1ab794446b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:20.316828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 36/71] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:20.316828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b820d3-10ef-415b-8811-1ab794446b12 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:20.362961 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:20.371877 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efd76f-cfad-41fc-bf8f-943eebe4dcd4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:20.423018 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1824dc-393f-454d-a048-30af0293752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:20.424047 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1824dc-393f-454d-a048-30af0293752e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:20.424522 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1824dc-393f-454d-a048-30af0293752e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:20.424985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1824dc-393f-454d-a048-30af0293752e None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:20.426341 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1824dc-393f-454d-a048-30af0293752e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:20.426839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1824dc-393f-454d-a048-30af0293752e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:20.429208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1824dc-393f-454d-a048-30af0293752e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:20.442006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 36/72] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:15:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:20.443937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61daaa-95e2-4e9e-8c31-28be248cbc67 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:20.444415 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61daaa-95e2-4e9e-8c31-28be248cbc67 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:20.444874 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61daaa-95e2-4e9e-8c31-28be248cbc67 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:20.446685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 37/73] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:20.449033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca4e73-9172-409f-b9b5-cd9f8d4fbe16 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:20.449789 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca4e73-9172-409f-b9b5-cd9f8d4fbe16 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:20.450528 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca4e73-9172-409f-b9b5-cd9f8d4fbe16 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:20.457912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 38/74] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:20.457912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca4e73-9172-409f-b9b5-cd9f8d4fbe16 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:20.477958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b820d3-10ef-415b-8811-1ab794446b12 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:20.496538 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:20.521901 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b34531-a696-41ff-b1d1-4160229ef9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:20.522675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b34531-a696-41ff-b1d1-4160229ef9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:20.523106 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b34531-a696-41ff-b1d1-4160229ef9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:20.523498 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b34531-a696-41ff-b1d1-4160229ef9a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:20.586140 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca4e73-9172-409f-b9b5-cd9f8d4fbe16 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:20.607583 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b34531-a696-41ff-b1d1-4160229ef9a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:20.609901 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b34531-a696-41ff-b1d1-4160229ef9a3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:20.628225 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226409b-c825-4102-bcc4-e2ae1635f98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:20.629218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226409b-c825-4102-bcc4-e2ae1635f98e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:20.629589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226409b-c825-4102-bcc4-e2ae1635f98e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:20.629925 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226409b-c825-4102-bcc4-e2ae1635f98e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:20.643667 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5226409b-c825-4102-bcc4-e2ae1635f98e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:20.653089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-5226409b-c825-4102-bcc4-e2ae1635f98e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:20.653089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:20.653089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:20.653089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:20.653089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:20.653089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:20.653089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:20.653089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:20.653089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:20.653089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:20.653089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:20.653089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:20.653089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:20.653089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:20.653089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:20.653089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:20.655005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:20.655005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:20.655005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:20.655005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:20.655005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:20.655005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:20.655005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:20.655005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:20.655005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:20.655005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:20.655005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:20.655005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:15:20.655005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:20.655005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:20.655005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:20.655005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:20.664605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:20.664605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:20.664605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:20.664605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:20.664605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:20.664605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:20.664605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:20.664605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:20.664605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:20.664605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:20.664605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:20.664605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:15:20.664605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 39/75] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:15:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:20.664605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226409b-c825-4102-bcc4-e2ae1635f98e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:20.683283 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:20.737276 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 37/76] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:15:20.738636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf1766-75d1-4a78-9cfd-8ed5520b1b2a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:20.738920 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf1766-75d1-4a78-9cfd-8ed5520b1b2a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:20.739208 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf1766-75d1-4a78-9cfd-8ed5520b1b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:20.757413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-11bf1766-75d1-4a78-9cfd-8ed5520b1b2a None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:20.758635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-11bf1766-75d1-4a78-9cfd-8ed5520b1b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:20.769587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:20.804286 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226409b-c825-4102-bcc4-e2ae1635f98e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:20.839096 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 38/77] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:20.840322 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0884b2b-0c52-4f02-97df-6f591e303188 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:20.840614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0884b2b-0c52-4f02-97df-6f591e303188 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:20.840897 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0884b2b-0c52-4f02-97df-6f591e303188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:20.868517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76ffb5-d600-48d7-807e-25d82339e313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:20.869447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76ffb5-d600-48d7-807e-25d82339e313 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:20.869746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76ffb5-d600-48d7-807e-25d82339e313 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:20.870029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76ffb5-d600-48d7-807e-25d82339e313 None admin] PATH_INFO: `/v3/users/b8be4e8c78444c25a8bd96f5851b92a1` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:20.871207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f0884b2b-0c52-4f02-97df-6f591e303188 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:20.871397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76ffb5-d600-48d7-807e-25d82339e313 None admin] RBAC: Authorizing `identity:get_user(user_id=b8be4e8c78444c25a8bd96f5851b92a1)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:20.872616 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f0884b2b-0c52-4f02-97df-6f591e303188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:20.872776 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76ffb5-d600-48d7-807e-25d82339e313 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:20.875263 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 40/78] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:20 2022] GET /identity/v3/users/b8be4e8c78444c25a8bd96f5851b92a1 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:20.876008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76ffb5-d600-48d7-807e-25d82339e313 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:20.956268 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 39/79] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:20.957866 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0884b2b-0c52-4f02-97df-6f591e303188 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:21.012703 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:21.037004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76ffb5-d600-48d7-807e-25d82339e313 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:21.063561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:21.087428 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bada3-d18e-4a9b-8fa2-9582d139eb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:21.088471 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bada3-d18e-4a9b-8fa2-9582d139eb41 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:21.088941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bada3-d18e-4a9b-8fa2-9582d139eb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:21.089499 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bada3-d18e-4a9b-8fa2-9582d139eb41 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:21.090699 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67bada3-d18e-4a9b-8fa2-9582d139eb41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:21.091151 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67bada3-d18e-4a9b-8fa2-9582d139eb41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:21.094082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67bada3-d18e-4a9b-8fa2-9582d139eb41 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:21.099062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0884b2b-0c52-4f02-97df-6f591e303188 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:21.104351 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 41/80] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:21.105808 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bada3-d18e-4a9b-8fa2-9582d139eb41 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:21.151760 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e213ac-8a8b-4854-92d8-4b669246e557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:21.153194 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e213ac-8a8b-4854-92d8-4b669246e557 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:21.153704 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e213ac-8a8b-4854-92d8-4b669246e557 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:21.154126 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e213ac-8a8b-4854-92d8-4b669246e557 None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:21.156359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e213ac-8a8b-4854-92d8-4b669246e557 None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:21.158925 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e213ac-8a8b-4854-92d8-4b669246e557 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:21.162025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 40/81] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:20 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:21.163118 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e213ac-8a8b-4854-92d8-4b669246e557 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:21.257315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bada3-d18e-4a9b-8fa2-9582d139eb41 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:21.304410 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec6453-1565-47a8-86fa-7b4c52238135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:21.305722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec6453-1565-47a8-86fa-7b4c52238135 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:21.306117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec6453-1565-47a8-86fa-7b4c52238135 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:21.306537 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec6453-1565-47a8-86fa-7b4c52238135 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:21.308566 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec6453-1565-47a8-86fa-7b4c52238135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:21.309346 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e213ac-8a8b-4854-92d8-4b669246e557 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:21.311141 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec6453-1565-47a8-86fa-7b4c52238135 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:21.313819 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 42/82] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:15:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:21.315208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec6453-1565-47a8-86fa-7b4c52238135 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:21.340574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:21.364635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89fa44-a4a2-41a6-af01-a180322a7982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:21.365473 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89fa44-a4a2-41a6-af01-a180322a7982 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:21.365876 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89fa44-a4a2-41a6-af01-a180322a7982 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:21.366258 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89fa44-a4a2-41a6-af01-a180322a7982 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:21.381644 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:21.452333 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89fa44-a4a2-41a6-af01-a180322a7982 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:21.454709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89fa44-a4a2-41a6-af01-a180322a7982 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:21.462590 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec6453-1565-47a8-86fa-7b4c52238135 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:21.503169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4663ef3-71ba-43c9-bab7-1d4c03a6325b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:21.503993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4663ef3-71ba-43c9-bab7-1d4c03a6325b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:21.504282 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4663ef3-71ba-43c9-bab7-1d4c03a6325b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:21.504568 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4663ef3-71ba-43c9-bab7-1d4c03a6325b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:21.517398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4663ef3-71ba-43c9-bab7-1d4c03a6325b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:21.525343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-e4663ef3-71ba-43c9-bab7-1d4c03a6325b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:21.525343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:21.525343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:21.525343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:21.525343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:21.525343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:21.525343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:21.525343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:21.525343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:21.525343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:21.525343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:21.525343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:21.525343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:21.525343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:21.525343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:21.525343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:21.527244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:21.527244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:21.527244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:21.527244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:21.527244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:21.527244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:21.527244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:21.527244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:21.527244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:21.527244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:21.527244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:21.527244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:15:21.527244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:21.527244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:21.527244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:21.527244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:21.528897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:21.528897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:21.528897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:21.528897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:21.528897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:21.528897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:21.528897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:21.528897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:21.528897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:21.528897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:21.528897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:21.528897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:15:21.528897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 43/83] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:21.528897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4663ef3-71ba-43c9-bab7-1d4c03a6325b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:21.583426 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 41/84] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:15:21.584298 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89fa44-a4a2-41a6-af01-a180322a7982 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:21.618382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:21.637628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:21.649143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4663ef3-71ba-43c9-bab7-1d4c03a6325b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:21.687959 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5891d-26fd-41c7-94fb-490ed7a588b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:21.689010 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5891d-26fd-41c7-94fb-490ed7a588b7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:21.689332 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5891d-26fd-41c7-94fb-490ed7a588b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:21.689613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5891d-26fd-41c7-94fb-490ed7a588b7 None admin] PATH_INFO: `/v3/users/5a11fde32c844e228722284109f82f80` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:21.690899 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5891d-26fd-41c7-94fb-490ed7a588b7 None admin] RBAC: Authorizing `identity:get_user(user_id=5a11fde32c844e228722284109f82f80)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:21.692420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5891d-26fd-41c7-94fb-490ed7a588b7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:21.694185 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 44/85] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:21 2022] GET /identity/v3/users/5a11fde32c844e228722284109f82f80 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:21.694967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5891d-26fd-41c7-94fb-490ed7a588b7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:21.722532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89fa44-a4a2-41a6-af01-a180322a7982 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:21.766636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aafaed-10b0-430e-867b-83fb3c63f15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:21.767205 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aafaed-10b0-430e-867b-83fb3c63f15b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:21.767495 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aafaed-10b0-430e-867b-83fb3c63f15b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:21.767778 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aafaed-10b0-430e-867b-83fb3c63f15b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:21.768679 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aafaed-10b0-430e-867b-83fb3c63f15b None admin] RBAC: Adding query filter params (role.id=ed6cf002b70f422190946ebcb114da60, scope.project.id=fda8b64de40a4fd698a2458618d1a582, user.id=b8be4e8c78444c25a8bd96f5851b92a1) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:21.769003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aafaed-10b0-430e-867b-83fb3c63f15b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:21.770635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aafaed-10b0-430e-867b-83fb3c63f15b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:21.777362 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 42/86] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:15:21 2022] GET /identity/v3/role_assignments?user.id=b8be4e8c78444c25a8bd96f5851b92a1&scope.project.id=fda8b64de40a4fd698a2458618d1a582&role.id=ed6cf002b70f422190946ebcb114da60 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:21.789861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aafaed-10b0-430e-867b-83fb3c63f15b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:21.851556 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5891d-26fd-41c7-94fb-490ed7a588b7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:21.897904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ea5c4-cb7b-4c71-ac7f-75b0ba024a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:21.898754 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ea5c4-cb7b-4c71-ac7f-75b0ba024a17 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:21.899056 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ea5c4-cb7b-4c71-ac7f-75b0ba024a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:21.899338 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ea5c4-cb7b-4c71-ac7f-75b0ba024a17 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:21.900114 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ea5c4-cb7b-4c71-ac7f-75b0ba024a17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:21.901997 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ea5c4-cb7b-4c71-ac7f-75b0ba024a17 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:21.912667 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aafaed-10b0-430e-867b-83fb3c63f15b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:21.922974 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:21.926058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:21.934787 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 45/87] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:15:21 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:15:21.947563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ea5c4-cb7b-4c71-ac7f-75b0ba024a17 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:21.986848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948f423-0497-4bae-a6d5-8d4cc2dc16e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:21.989376 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948f423-0497-4bae-a6d5-8d4cc2dc16e9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:21.990063 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948f423-0497-4bae-a6d5-8d4cc2dc16e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:21.990682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948f423-0497-4bae-a6d5-8d4cc2dc16e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:21.992862 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948f423-0497-4bae-a6d5-8d4cc2dc16e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:21.994165 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948f423-0497-4bae-a6d5-8d4cc2dc16e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:21.998055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948f423-0497-4bae-a6d5-8d4cc2dc16e9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:22.023712 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 43/88] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:15:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:22.025575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948f423-0497-4bae-a6d5-8d4cc2dc16e9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:22.118701 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ea5c4-cb7b-4c71-ac7f-75b0ba024a17 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:22.186795 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948f423-0497-4bae-a6d5-8d4cc2dc16e9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:22.187313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd41e3e-2ae2-4f16-a798-fbae11c9a20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:22.188714 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd41e3e-2ae2-4f16-a798-fbae11c9a20b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:22.188986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd41e3e-2ae2-4f16-a798-fbae11c9a20b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:22.189436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd41e3e-2ae2-4f16-a798-fbae11c9a20b None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:22.191916 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd41e3e-2ae2-4f16-a798-fbae11c9a20b None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:22.194994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd41e3e-2ae2-4f16-a798-fbae11c9a20b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:22.199163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 46/89] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:21 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:22.200783 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd41e3e-2ae2-4f16-a798-fbae11c9a20b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:22.245634 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a863e6-3585-4e1d-9c67-5eb07c295039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:22.246760 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a863e6-3585-4e1d-9c67-5eb07c295039 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:22.247271 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a863e6-3585-4e1d-9c67-5eb07c295039 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:22.247781 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a863e6-3585-4e1d-9c67-5eb07c295039 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:22.314925 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:22.318842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:22.363136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a863e6-3585-4e1d-9c67-5eb07c295039 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:22.364873 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a863e6-3585-4e1d-9c67-5eb07c295039 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:22.402652 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd41e3e-2ae2-4f16-a798-fbae11c9a20b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:22.472829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dd135-31be-436a-b9d8-ae8729c565a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:22.474734 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dd135-31be-436a-b9d8-ae8729c565a6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:22.475549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dd135-31be-436a-b9d8-ae8729c565a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:22.476143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dd135-31be-436a-b9d8-ae8729c565a6 None admin] PATH_INFO: `/v3/users/b8be4e8c78444c25a8bd96f5851b92a1` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:22.478776 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016dd135-31be-436a-b9d8-ae8729c565a6 None admin] RBAC: Authorizing `identity:get_user(user_id=b8be4e8c78444c25a8bd96f5851b92a1)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:22.481922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016dd135-31be-436a-b9d8-ae8729c565a6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:22.484934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 47/90] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:22 2022] GET /identity/v3/users/b8be4e8c78444c25a8bd96f5851b92a1 => generated 275 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:22.486789 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dd135-31be-436a-b9d8-ae8729c565a6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:22.545272 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 44/91] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:15:22.546468 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a863e6-3585-4e1d-9c67-5eb07c295039 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:22.638355 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dd135-31be-436a-b9d8-ae8729c565a6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:22.692338 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:22.695336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f043532-8aae-4064-bbf5-43c6b6370827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:22.696318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043532-8aae-4064-bbf5-43c6b6370827 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:22.697793 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043532-8aae-4064-bbf5-43c6b6370827 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:22.698928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043532-8aae-4064-bbf5-43c6b6370827 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:22.702195 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f043532-8aae-4064-bbf5-43c6b6370827 None admin] RBAC: Adding query filter params (role.id=6e5dce7d61b34704ad2cd32251fa4d34, scope.project.id=fda8b64de40a4fd698a2458618d1a582, user.id=b8be4e8c78444c25a8bd96f5851b92a1) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:22.703315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f043532-8aae-4064-bbf5-43c6b6370827 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:22.705031 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f043532-8aae-4064-bbf5-43c6b6370827 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:22.714006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:22.721591 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 48/92] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:15:22 2022] GET /identity/v3/role_assignments?user.id=b8be4e8c78444c25a8bd96f5851b92a1&scope.project.id=fda8b64de40a4fd698a2458618d1a582&role.id=6e5dce7d61b34704ad2cd32251fa4d34 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:22.723811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f043532-8aae-4064-bbf5-43c6b6370827 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:22.783309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a863e6-3585-4e1d-9c67-5eb07c295039 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:22.850731 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a9177-f924-43fb-a268-cbf807f84990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:22.851926 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a9177-f924-43fb-a268-cbf807f84990 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:22.852439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a9177-f924-43fb-a268-cbf807f84990 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:22.852916 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a9177-f924-43fb-a268-cbf807f84990 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:22.973561 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f043532-8aae-4064-bbf5-43c6b6370827 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:23.027671 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a9177-f924-43fb-a268-cbf807f84990 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:23.031672 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a9177-f924-43fb-a268-cbf807f84990 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:23.094891 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33dbc9-0519-424c-bb8f-e4663a45a37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:23.096317 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33dbc9-0519-424c-bb8f-e4663a45a37f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:23.096790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33dbc9-0519-424c-bb8f-e4663a45a37f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:23.097257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33dbc9-0519-424c-bb8f-e4663a45a37f None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:23.099516 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33dbc9-0519-424c-bb8f-e4663a45a37f None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:23.102191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33dbc9-0519-424c-bb8f-e4663a45a37f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:23.105487 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 49/93] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:22 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:23.115588 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33dbc9-0519-424c-bb8f-e4663a45a37f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:23.178737 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:23.256330 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 45/94] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:15:23.263684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a9177-f924-43fb-a268-cbf807f84990 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:23.359769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:23.492865 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33dbc9-0519-424c-bb8f-e4663a45a37f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:23.511357 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a9177-f924-43fb-a268-cbf807f84990 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:23.592392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bc097-870f-4419-97ec-e8e8e6fed567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:23.594691 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bc097-870f-4419-97ec-e8e8e6fed567 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:23.595150 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bc097-870f-4419-97ec-e8e8e6fed567 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:23.595619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bc097-870f-4419-97ec-e8e8e6fed567 None admin] PATH_INFO: `/v3/projects/0cf91e07189443cfb23d8c72d10a1e44` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:23.598008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bc097-870f-4419-97ec-e8e8e6fed567 None admin] RBAC: Authorizing `identity:get_project(project_id=0cf91e07189443cfb23d8c72d10a1e44)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:23.601071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bc097-870f-4419-97ec-e8e8e6fed567 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:23.604840 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 46/95] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:23 2022] GET /identity/v3/projects/0cf91e07189443cfb23d8c72d10a1e44 => generated 312 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:23.610655 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b8ab8-3891-4e3a-9cf2-cb460a93d27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:23.611825 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b8ab8-3891-4e3a-9cf2-cb460a93d27a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:23.612313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b8ab8-3891-4e3a-9cf2-cb460a93d27a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:23.612811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b8ab8-3891-4e3a-9cf2-cb460a93d27a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:23.615712 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bc097-870f-4419-97ec-e8e8e6fed567 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:23.621061 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b8ab8-3891-4e3a-9cf2-cb460a93d27a None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=fda8b64de40a4fd698a2458618d1a582, user.id=5a11fde32c844e228722284109f82f80) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:23.621695 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b8ab8-3891-4e3a-9cf2-cb460a93d27a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:23.625199 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b8ab8-3891-4e3a-9cf2-cb460a93d27a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:23.664699 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 50/96] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:15:23 2022] GET /identity/v3/role_assignments?user.id=5a11fde32c844e228722284109f82f80&scope.project.id=fda8b64de40a4fd698a2458618d1a582&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 259 bytes in 550 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:23.719341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:23.856496 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bc097-870f-4419-97ec-e8e8e6fed567 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:23.931324 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d489083-a9e2-4d95-bf73-ff535f489b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:23.932510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d489083-a9e2-4d95-bf73-ff535f489b13 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:23.933035 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d489083-a9e2-4d95-bf73-ff535f489b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:23.933580 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d489083-a9e2-4d95-bf73-ff535f489b13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:23.935334 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d489083-a9e2-4d95-bf73-ff535f489b13 None admin] RBAC: Adding query filter params (role.id=ed6cf002b70f422190946ebcb114da60, scope.project.id=0cf91e07189443cfb23d8c72d10a1e44, user.id=b8be4e8c78444c25a8bd96f5851b92a1) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:23.935918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d489083-a9e2-4d95-bf73-ff535f489b13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:23.938752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d489083-a9e2-4d95-bf73-ff535f489b13 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:23.954655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 47/97] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:15:23 2022] GET /identity/v3/role_assignments?user.id=b8be4e8c78444c25a8bd96f5851b92a1&scope.project.id=0cf91e07189443cfb23d8c72d10a1e44&role.id=ed6cf002b70f422190946ebcb114da60 => generated 259 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:25.892107 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7d266-94cb-4e7c-bbf6-e1dca2b7fa1b None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:25.892652 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7d266-94cb-4e7c-bbf6-e1dca2b7fa1b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:25.893121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7d266-94cb-4e7c-bbf6-e1dca2b7fa1b None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:25.896059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 51/98] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:25.903265 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b9f26-53dc-441c-938b-41aa2c4cdcd1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:25.903775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b9f26-53dc-441c-938b-41aa2c4cdcd1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:25.904290 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b9f26-53dc-441c-938b-41aa2c4cdcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:25.942107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-692b9f26-53dc-441c-938b-41aa2c4cdcd1 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:25.944325 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-692b9f26-53dc-441c-938b-41aa2c4cdcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:26.098068 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 48/99] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:26.134437 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c7d266-94cb-4e7c-bbf6-e1dca2b7fa1b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:26.401774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c7d266-94cb-4e7c-bbf6-e1dca2b7fa1b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:26.484994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:26.497887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fb035-e75c-47ac-8102-2350403b3198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:26.499365 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fb035-e75c-47ac-8102-2350403b3198 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:26.499859 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fb035-e75c-47ac-8102-2350403b3198 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:26.500364 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fb035-e75c-47ac-8102-2350403b3198 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:26.531811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fb035-e75c-47ac-8102-2350403b3198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:26.552400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-8f8fb035-e75c-47ac-8102-2350403b3198 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:15:26.552400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:26.552400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:26.552400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:26.552400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:26.552400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:26.552400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:26.552400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:26.552400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:26.552400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:26.552400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:26.552400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:26.552400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:26.552400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:26.552400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:26.552400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:26.554121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:26.554121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:26.554121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:26.554121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:26.554121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:26.554121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:26.554121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:26.554121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:26.554121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:26.554121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:26.554121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:26.554121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:15:26.554121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:26.554121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:26.554121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:26.554121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:26.555629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:26.555629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:26.555629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:26.555629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:26.555629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:26.555629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:26.555629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:26.555629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:26.555629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:26.555629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:26.555629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:15:26.555629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:15:26.555629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 52/100] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:26.567346 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b9f26-53dc-441c-938b-41aa2c4cdcd1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:26.606977 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb24de2-23a9-4542-b1ff-e86beacdedb4 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:26.607400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb24de2-23a9-4542-b1ff-e86beacdedb4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:26.607877 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb24de2-23a9-4542-b1ff-e86beacdedb4 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:26.611623 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 53/101] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:15:26.620484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1ab22-32ea-4bea-a009-44ca6b7e7d8b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:26.620947 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1ab22-32ea-4bea-a009-44ca6b7e7d8b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:26.621462 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1ab22-32ea-4bea-a009-44ca6b7e7d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:26.641087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:26.654887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-aee1ab22-32ea-4bea-a009-44ca6b7e7d8b None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:26.657995 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-aee1ab22-32ea-4bea-a009-44ca6b7e7d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:26.850985 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b9f26-53dc-441c-938b-41aa2c4cdcd1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:26.881018 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 54/102] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:15:26.883223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d8b8f-5d3b-4a5b-9fb1-5d20a9b2d394 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:26.883740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d8b8f-5d3b-4a5b-9fb1-5d20a9b2d394 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:26.884245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d8b8f-5d3b-4a5b-9fb1-5d20a9b2d394 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:26.886809 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 55/103] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:26.896428 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566a181-69a4-406f-9a0a-a95861e8056e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:26.897382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566a181-69a4-406f-9a0a-a95861e8056e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:26.897538 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566a181-69a4-406f-9a0a-a95861e8056e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:26.942448 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-7566a181-69a4-406f-9a0a-a95861e8056e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:26.942696 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3e3a7-da19-4470-a102-17f934636156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:26.943585 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3e3a7-da19-4470-a102-17f934636156 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:26.944198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3e3a7-da19-4470-a102-17f934636156 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:26.944440 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7566a181-69a4-406f-9a0a-a95861e8056e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:26.944690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3e3a7-da19-4470-a102-17f934636156 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:26.946039 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3e3a7-da19-4470-a102-17f934636156 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:26.951116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3e3a7-da19-4470-a102-17f934636156 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:26.954358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3e3a7-da19-4470-a102-17f934636156 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:26.978603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 49/104] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:15:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:26.986834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3e3a7-da19-4470-a102-17f934636156 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:27.160195 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 56/105] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:15:27.161759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566a181-69a4-406f-9a0a-a95861e8056e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:27.166217 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:27.187387 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3e3a7-da19-4470-a102-17f934636156 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:27.254483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce13c3a-dedb-4662-a567-f0af1005e15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:27.255726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce13c3a-dedb-4662-a567-f0af1005e15c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:27.256161 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce13c3a-dedb-4662-a567-f0af1005e15c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:27.256586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce13c3a-dedb-4662-a567-f0af1005e15c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:27.276021 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce13c3a-dedb-4662-a567-f0af1005e15c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:27.296613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-8ce13c3a-dedb-4662-a567-f0af1005e15c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:27.296613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:27.296613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:27.296613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:27.296613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:27.296613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:27.296613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:27.296613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:27.296613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:27.296613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:27.296613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:27.296613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:27.296613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:27.296613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:27.296613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:27.296613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:27.298309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:27.298309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:27.298309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:27.298309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:27.298309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:27.298309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:27.298309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:27.298309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:27.298309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:27.298309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:27.298309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:27.298309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:15:27.298309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:27.298309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:27.298309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:27.298309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:27.299754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:27.299754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:27.299754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:27.299754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:27.299754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:27.299754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:27.299754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:27.299754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:27.299754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:27.299754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:27.299754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:27.299754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:15:27.303970 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 50/106] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:27.305814 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce13c3a-dedb-4662-a567-f0af1005e15c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:27.361881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:27.485763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566a181-69a4-406f-9a0a-a95861e8056e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:27.578773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5ce57-94ba-4c36-a240-e0a6a928e6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:27.580666 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5ce57-94ba-4c36-a240-e0a6a928e6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:27.581384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5ce57-94ba-4c36-a240-e0a6a928e6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:27.582041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5ce57-94ba-4c36-a240-e0a6a928e6a4 None admin] PATH_INFO: `/v3/users/ff214e62139649ca9b5b266907aa8e2c` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:27.583621 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce13c3a-dedb-4662-a567-f0af1005e15c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:27.610838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5ce57-94ba-4c36-a240-e0a6a928e6a4 None admin] RBAC: Authorizing `identity:get_user(user_id=ff214e62139649ca9b5b266907aa8e2c)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:27.612854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5ce57-94ba-4c36-a240-e0a6a928e6a4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:27.618612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 57/107] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:27 2022] GET /identity/v3/users/ff214e62139649ca9b5b266907aa8e2c => generated 289 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:27.622204 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5ce57-94ba-4c36-a240-e0a6a928e6a4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:27.669794 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d741aa-3ec6-47e6-af9c-50edfceac7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:27.670904 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d741aa-3ec6-47e6-af9c-50edfceac7ca None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:27.671376 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d741aa-3ec6-47e6-af9c-50edfceac7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:27.671805 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d741aa-3ec6-47e6-af9c-50edfceac7ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:27.684743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d741aa-3ec6-47e6-af9c-50edfceac7ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:27.693965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-00d741aa-3ec6-47e6-af9c-50edfceac7ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:15:27.693965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:27.693965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:27.693965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:27.693965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:27.693965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:27.693965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:27.693965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:27.693965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:27.693965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:27.693965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:27.693965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:27.693965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:27.693965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:27.693965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:27.693965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:27.694900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:27.694900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:27.694900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:27.694900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:27.694900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:27.694900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:27.694900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:27.694900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:27.694900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:27.694900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:27.694900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:27.694900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:15:27.694900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:27.694900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:27.694900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:27.694900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:27.695662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:27.695662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:27.695662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:27.695662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:27.695662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:27.695662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:27.695662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:27.695662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:27.695662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:27.695662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:27.695662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:15:27.695662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:15:27.698343 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 51/108] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:15:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:27.699943 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc61975-b193-483a-bec5-c217baea4392 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:27.700446 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc61975-b193-483a-bec5-c217baea4392 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:27.700961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc61975-b193-483a-bec5-c217baea4392 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:27.703112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 52/109] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:27.704103 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc61975-b193-483a-bec5-c217baea4392 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:27.729511 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:27.774193 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5ce57-94ba-4c36-a240-e0a6a928e6a4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:27.789487 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:27.852892 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc0981-86e5-453f-a6c0-d05c1d93a38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:27.853898 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc0981-86e5-453f-a6c0-d05c1d93a38b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:27.854390 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc0981-86e5-453f-a6c0-d05c1d93a38b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:27.854834 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc0981-86e5-453f-a6c0-d05c1d93a38b None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:27.855788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc0981-86e5-453f-a6c0-d05c1d93a38b None admin] RBAC: Adding query filter params (name=member) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:27.856245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc0981-86e5-453f-a6c0-d05c1d93a38b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:27.857850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc0981-86e5-453f-a6c0-d05c1d93a38b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:27.868082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 58/110] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:15:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:27.869272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc0981-86e5-453f-a6c0-d05c1d93a38b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:27.910376 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc61975-b193-483a-bec5-c217baea4392 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:27.985039 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01420011-dd83-4a04-a7c5-fbff648a4bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:27.986155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01420011-dd83-4a04-a7c5-fbff648a4bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:27.986641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01420011-dd83-4a04-a7c5-fbff648a4bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:27.987106 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01420011-dd83-4a04-a7c5-fbff648a4bf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:28.000034 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc0981-86e5-453f-a6c0-d05c1d93a38b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:28.043601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a8036-61eb-4e77-b9e9-02cca9fe6548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:28.044982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a8036-61eb-4e77-b9e9-02cca9fe6548 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:28.045518 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a8036-61eb-4e77-b9e9-02cca9fe6548 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:28.046420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a8036-61eb-4e77-b9e9-02cca9fe6548 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:28.047717 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a8036-61eb-4e77-b9e9-02cca9fe6548 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:28.048239 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a8036-61eb-4e77-b9e9-02cca9fe6548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:28.049979 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a8036-61eb-4e77-b9e9-02cca9fe6548 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:28.060350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 59/111] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:15:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:28.062087 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202c476-789d-4580-aaad-8bcc42c35e20 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:28.062730 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202c476-789d-4580-aaad-8bcc42c35e20 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:28.063289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202c476-789d-4580-aaad-8bcc42c35e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:28.084758 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d202c476-789d-4580-aaad-8bcc42c35e20 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:28.086672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d202c476-789d-4580-aaad-8bcc42c35e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:28.120795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:28.128535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01420011-dd83-4a04-a7c5-fbff648a4bf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:28.131386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01420011-dd83-4a04-a7c5-fbff648a4bf2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:28.160272 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:28.182013 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 60/112] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:28.183582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202c476-789d-4580-aaad-8bcc42c35e20 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:28.322883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202c476-789d-4580-aaad-8bcc42c35e20 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:28.328241 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 53/113] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:15:28.329962 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01420011-dd83-4a04-a7c5-fbff648a4bf2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:28.367467 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2a488-261d-46ce-82f0-26396b562e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:28.368639 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2a488-261d-46ce-82f0-26396b562e76 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:28.369159 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2a488-261d-46ce-82f0-26396b562e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:28.370114 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2a488-261d-46ce-82f0-26396b562e76 None admin] PATH_INFO: `/v3/users/b8be4e8c78444c25a8bd96f5851b92a1` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:28.371658 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2a488-261d-46ce-82f0-26396b562e76 None admin] RBAC: Authorizing `identity:get_user(user_id=b8be4e8c78444c25a8bd96f5851b92a1)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:28.373448 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2a488-261d-46ce-82f0-26396b562e76 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:28.376658 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 61/114] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:28 2022] GET /identity/v3/users/b8be4e8c78444c25a8bd96f5851b92a1 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:28.377906 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2a488-261d-46ce-82f0-26396b562e76 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:28.422558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:28.522470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01420011-dd83-4a04-a7c5-fbff648a4bf2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:28.524949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2a488-261d-46ce-82f0-26396b562e76 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:28.538913 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:28.564543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15837b8e-a9b0-4cea-b42c-87d771b3dc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:28.565718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15837b8e-a9b0-4cea-b42c-87d771b3dc90 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:28.566162 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15837b8e-a9b0-4cea-b42c-87d771b3dc90 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:28.566610 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15837b8e-a9b0-4cea-b42c-87d771b3dc90 None admin] PATH_INFO: `/v3/users/b8be4e8c78444c25a8bd96f5851b92a1` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:28.568043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15837b8e-a9b0-4cea-b42c-87d771b3dc90 None admin] RBAC: Authorizing `identity:get_user(user_id=b8be4e8c78444c25a8bd96f5851b92a1)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:28.569721 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15837b8e-a9b0-4cea-b42c-87d771b3dc90 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:28.571584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 54/115] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:28 2022] GET /identity/v3/users/b8be4e8c78444c25a8bd96f5851b92a1 => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:28.576798 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15837b8e-a9b0-4cea-b42c-87d771b3dc90 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:28.594920 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24add910-fa4a-43d9-bf77-31b10a08fec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:28.596643 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24add910-fa4a-43d9-bf77-31b10a08fec4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:28.597708 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24add910-fa4a-43d9-bf77-31b10a08fec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:28.598242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24add910-fa4a-43d9-bf77-31b10a08fec4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:28.599832 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24add910-fa4a-43d9-bf77-31b10a08fec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:28.601751 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24add910-fa4a-43d9-bf77-31b10a08fec4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:28.604014 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 62/116] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:15:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:28.605946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24add910-fa4a-43d9-bf77-31b10a08fec4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:28.694760 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15837b8e-a9b0-4cea-b42c-87d771b3dc90 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:28.732039 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e136646-30ab-43ba-b889-d178cc383fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:28.732930 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e136646-30ab-43ba-b889-d178cc383fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:28.733220 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e136646-30ab-43ba-b889-d178cc383fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:28.734054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e136646-30ab-43ba-b889-d178cc383fc1 None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:28.735356 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e136646-30ab-43ba-b889-d178cc383fc1 None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:28.736844 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e136646-30ab-43ba-b889-d178cc383fc1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:28.738805 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 55/117] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:28 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:28.739444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e136646-30ab-43ba-b889-d178cc383fc1 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:28.754214 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:28.762041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24add910-fa4a-43d9-bf77-31b10a08fec4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:28.793178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:28.817392 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd69eb2-37fc-4a77-8d35-db62f5e3420b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:28.818279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd69eb2-37fc-4a77-8d35-db62f5e3420b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:28.818566 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd69eb2-37fc-4a77-8d35-db62f5e3420b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:28.818847 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd69eb2-37fc-4a77-8d35-db62f5e3420b None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:28.820118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd69eb2-37fc-4a77-8d35-db62f5e3420b None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:28.821619 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd69eb2-37fc-4a77-8d35-db62f5e3420b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:28.829614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 63/118] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:28 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:28.829614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669e0d5-492e-41c0-953c-879af6ba7fbf None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:28.829614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669e0d5-492e-41c0-953c-879af6ba7fbf None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:28.829614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669e0d5-492e-41c0-953c-879af6ba7fbf None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:28.829614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 64/119] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:28.829614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669e0d5-492e-41c0-953c-879af6ba7fbf None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:28.877894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e136646-30ab-43ba-b889-d178cc383fc1 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:28.924839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267478e6-4f80-4fbd-9a8f-8d234e2251f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:28.926382 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267478e6-4f80-4fbd-9a8f-8d234e2251f3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:28.926932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267478e6-4f80-4fbd-9a8f-8d234e2251f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:28.927471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267478e6-4f80-4fbd-9a8f-8d234e2251f3 None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:28.929490 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267478e6-4f80-4fbd-9a8f-8d234e2251f3 None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:28.932119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267478e6-4f80-4fbd-9a8f-8d234e2251f3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:28.934870 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 56/120] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:28 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:28.935970 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267478e6-4f80-4fbd-9a8f-8d234e2251f3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:28.975895 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669e0d5-492e-41c0-953c-879af6ba7fbf None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:29.023918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809accb-f5e3-482a-9872-b04dfac9fb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:29.025205 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809accb-f5e3-482a-9872-b04dfac9fb0c None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:29.025663 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809accb-f5e3-482a-9872-b04dfac9fb0c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:29.027039 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809accb-f5e3-482a-9872-b04dfac9fb0c None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:29.028050 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9809accb-f5e3-482a-9872-b04dfac9fb0c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:29.029843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9809accb-f5e3-482a-9872-b04dfac9fb0c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:29.065958 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 65/121] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 06:15:28 2022] POST /identity/v3/projects => generated 302 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:15:29.066833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809accb-f5e3-482a-9872-b04dfac9fb0c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:29.067236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267478e6-4f80-4fbd-9a8f-8d234e2251f3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:29.098836 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:29.113246 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:29.134107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffcb38d-2543-4394-882a-cceaeabf71d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:29.134805 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcb38d-2543-4394-882a-cceaeabf71d8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:29.135106 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcb38d-2543-4394-882a-cceaeabf71d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:29.135405 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcb38d-2543-4394-882a-cceaeabf71d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:29.136290 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffcb38d-2543-4394-882a-cceaeabf71d8 None admin] RBAC: Adding query filter params (role.id=f4272c5247654dfdbd6abccbd56374ff, scope.project.id=fda8b64de40a4fd698a2458618d1a582, user.id=ff214e62139649ca9b5b266907aa8e2c) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:29.136619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffcb38d-2543-4394-882a-cceaeabf71d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:29.138437 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffcb38d-2543-4394-882a-cceaeabf71d8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:29.145890 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 57/122] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:15:28 2022] GET /identity/v3/role_assignments?user.id=ff214e62139649ca9b5b266907aa8e2c&scope.project.id=fda8b64de40a4fd698a2458618d1a582&role.id=f4272c5247654dfdbd6abccbd56374ff => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:29.147072 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937e42a-fa9b-4149-963a-3adc4ee9bb1d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:29.147435 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937e42a-fa9b-4149-963a-3adc4ee9bb1d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:29.147642 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937e42a-fa9b-4149-963a-3adc4ee9bb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:29.168966 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2937e42a-fa9b-4149-963a-3adc4ee9bb1d None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:29.170329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2937e42a-fa9b-4149-963a-3adc4ee9bb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:29.257158 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 58/123] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:29.258067 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937e42a-fa9b-4149-963a-3adc4ee9bb1d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:29.367247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809accb-f5e3-482a-9872-b04dfac9fb0c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:29.379845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937e42a-fa9b-4149-963a-3adc4ee9bb1d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:29.394318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:29.450769 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587edf6-4ed7-472f-8c29-28bae8e1361c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:29.451033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b8f42-1a76-4e6d-8f51-c33d095d0f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:29.451692 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587edf6-4ed7-472f-8c29-28bae8e1361c None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:29.451965 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b8f42-1a76-4e6d-8f51-c33d095d0f2a None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:29.452098 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587edf6-4ed7-472f-8c29-28bae8e1361c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:29.452826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587edf6-4ed7-472f-8c29-28bae8e1361c None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582/users/b8be4e8c78444c25a8bd96f5851b92a1/roles/6e5dce7d61b34704ad2cd32251fa4d34` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:29.452968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b8f42-1a76-4e6d-8f51-c33d095d0f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:29.453091 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b8f42-1a76-4e6d-8f51-c33d095d0f2a None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582/users/b8be4e8c78444c25a8bd96f5851b92a1/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:29.456464 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587edf6-4ed7-472f-8c29-28bae8e1361c None admin] RBAC: Authorizing `identity:create_grant(project_id=fda8b64de40a4fd698a2458618d1a582, user_id=b8be4e8c78444c25a8bd96f5851b92a1, role_id=6e5dce7d61b34704ad2cd32251fa4d34)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:29.456823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b8f42-1a76-4e6d-8f51-c33d095d0f2a None admin] RBAC: Authorizing `identity:create_grant(project_id=fda8b64de40a4fd698a2458618d1a582, user_id=b8be4e8c78444c25a8bd96f5851b92a1, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:29.458191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587edf6-4ed7-472f-8c29-28bae8e1361c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:29.458346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b8f42-1a76-4e6d-8f51-c33d095d0f2a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:29.476257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 66/124] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:15:29 2022] PUT /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582/users/b8be4e8c78444c25a8bd96f5851b92a1/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:15:29.477243 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b8f42-1a76-4e6d-8f51-c33d095d0f2a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:29.481024 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 59/125] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:15:29 2022] PUT /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582/users/b8be4e8c78444c25a8bd96f5851b92a1/roles/6e5dce7d61b34704ad2cd32251fa4d34 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:15:29.567374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e8f1f-f1cb-425e-8c50-218e2188e779 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:29.567916 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e8f1f-f1cb-425e-8c50-218e2188e779 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:29.568309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e8f1f-f1cb-425e-8c50-218e2188e779 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:29.570282 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 60/126] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:15:29.579660 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69dee6-a60f-40bf-bb95-6b97fa82a749 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:29.580133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69dee6-a60f-40bf-bb95-6b97fa82a749 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:29.580373 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69dee6-a60f-40bf-bb95-6b97fa82a749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:29.606464 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:29.610940 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4d69dee6-a60f-40bf-bb95-6b97fa82a749 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:29.612509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4d69dee6-a60f-40bf-bb95-6b97fa82a749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:29.637595 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b8f42-1a76-4e6d-8f51-c33d095d0f2a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:29.690560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bd007-e214-496d-9c8b-aa52f597313b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:29.691388 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bd007-e214-496d-9c8b-aa52f597313b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:29.691683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bd007-e214-496d-9c8b-aa52f597313b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:29.691961 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bd007-e214-496d-9c8b-aa52f597313b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:29.707057 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bd007-e214-496d-9c8b-aa52f597313b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:29.716044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-dd3bd007-e214-496d-9c8b-aa52f597313b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:29.716044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:29.716044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:29.716044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:29.716044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:29.716044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:29.716044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:29.716044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:29.716044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:29.716044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:29.716044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:29.716044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:29.716044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:29.716044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:29.716044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:29.716044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:29.717929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:29.717929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:29.717929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:29.717929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:29.717929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:29.717929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:29.717929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:29.717929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:29.717929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:29.717929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:29.717929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:29.717929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:15:29.717929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:29.717929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:29.717929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:29.717929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:29.719968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:29.719968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:29.719968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:29.719968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:29.719968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:29.719968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:29.719968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:29.719968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:29.719968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:29.719968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:29.719968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:29.719968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:15:29.719968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 67/127] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:15:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:29.727987 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bd007-e214-496d-9c8b-aa52f597313b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:29.761543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 61/128] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:15:29.824261 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69dee6-a60f-40bf-bb95-6b97fa82a749 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:29.857143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bd007-e214-496d-9c8b-aa52f597313b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:29.912917 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1a074-740a-4a27-9ce0-e39f6cbec6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:29.913714 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1a074-740a-4a27-9ce0-e39f6cbec6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:29.914018 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1a074-740a-4a27-9ce0-e39f6cbec6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:29.914161 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1a074-740a-4a27-9ce0-e39f6cbec6b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:29.914905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1a074-740a-4a27-9ce0-e39f6cbec6b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:29.915221 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1a074-740a-4a27-9ce0-e39f6cbec6b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:29.916722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1a074-740a-4a27-9ce0-e39f6cbec6b5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:29.924963 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:29.932181 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 68/129] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:29.939840 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1a074-740a-4a27-9ce0-e39f6cbec6b5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:30.074148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1a074-740a-4a27-9ce0-e39f6cbec6b5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:30.093309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:30.101181 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69dee6-a60f-40bf-bb95-6b97fa82a749 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:30.115318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f154b90-9014-4568-bbab-b33999f09e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:30.116226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f154b90-9014-4568-bbab-b33999f09e4f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:30.116558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f154b90-9014-4568-bbab-b33999f09e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:30.116846 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f154b90-9014-4568-bbab-b33999f09e4f None admin] PATH_INFO: `/v3/users/5a11fde32c844e228722284109f82f80` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:30.118162 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f154b90-9014-4568-bbab-b33999f09e4f None admin] RBAC: Authorizing `identity:get_user(user_id=5a11fde32c844e228722284109f82f80)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:30.119646 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f154b90-9014-4568-bbab-b33999f09e4f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:30.121482 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 69/130] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:29 2022] GET /identity/v3/users/5a11fde32c844e228722284109f82f80 => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:30.127755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f154b90-9014-4568-bbab-b33999f09e4f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:30.214658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f5431-d0c8-4d0d-ae8f-00fc526227ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:30.216533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f5431-d0c8-4d0d-ae8f-00fc526227ff None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:30.217692 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f5431-d0c8-4d0d-ae8f-00fc526227ff None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:30.220685 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f5431-d0c8-4d0d-ae8f-00fc526227ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:30.255483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f5431-d0c8-4d0d-ae8f-00fc526227ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:30.259541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:30.276483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-bb2f5431-d0c8-4d0d-ae8f-00fc526227ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:30.276483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:30.276483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:30.276483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:30.276483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:30.276483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:30.276483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:30.276483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:30.276483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:30.276483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:30.276483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:30.276483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:30.276483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:30.276483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:30.276483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:30.276483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:30.278264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:30.278264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:30.278264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:30.278264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:30.278264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:30.278264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:30.278264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:30.278264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:30.278264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:30.278264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:30.278264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:30.278264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:15:30.278264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:30.278264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:30.278264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:30.278264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:30.279826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:30.279826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:30.279826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:30.279826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:30.279826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:30.279826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:30.279826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:30.279826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:30.279826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:30.279826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:30.279826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:30.279826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:15:30.285241 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 62/131] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 462 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:15:30.295244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f5431-d0c8-4d0d-ae8f-00fc526227ff None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:30.325320 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f154b90-9014-4568-bbab-b33999f09e4f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:30.362721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a29ea-04be-49cf-a246-2c058a5f3200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:30.364067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a29ea-04be-49cf-a246-2c058a5f3200 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:30.364067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a29ea-04be-49cf-a246-2c058a5f3200 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:30.364350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a29ea-04be-49cf-a246-2c058a5f3200 None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:30.365601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a29ea-04be-49cf-a246-2c058a5f3200 None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:30.367089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a29ea-04be-49cf-a246-2c058a5f3200 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:30.369140 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 70/132] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:30 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:30.378468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a29ea-04be-49cf-a246-2c058a5f3200 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:30.611132 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a29ea-04be-49cf-a246-2c058a5f3200 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:30.630207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f5431-d0c8-4d0d-ae8f-00fc526227ff None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:30.696374 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68dc63-1a1d-46ae-b785-71c5ea269ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:30.697319 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68dc63-1a1d-46ae-b785-71c5ea269ef4 None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:30.697589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68dc63-1a1d-46ae-b785-71c5ea269ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:30.697875 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68dc63-1a1d-46ae-b785-71c5ea269ef4 None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582/users/5a11fde32c844e228722284109f82f80/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:30.700868 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68dc63-1a1d-46ae-b785-71c5ea269ef4 None admin] RBAC: Authorizing `identity:create_grant(project_id=fda8b64de40a4fd698a2458618d1a582, user_id=5a11fde32c844e228722284109f82f80, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:30.706014 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68dc63-1a1d-46ae-b785-71c5ea269ef4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:30.709226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a3f69-c299-429b-9422-fd325f0f87d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:30.709951 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a3f69-c299-429b-9422-fd325f0f87d9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:30.710237 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a3f69-c299-429b-9422-fd325f0f87d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:30.710517 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a3f69-c299-429b-9422-fd325f0f87d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:30.711402 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a3f69-c299-429b-9422-fd325f0f87d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:30.711752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a3f69-c299-429b-9422-fd325f0f87d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:30.713176 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a3f69-c299-429b-9422-fd325f0f87d9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:30.718885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:30.732436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 71/133] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:15:30 2022] PUT /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582/users/5a11fde32c844e228722284109f82f80/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:15:30.739231 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 63/134] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:15:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:30.748089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68dc63-1a1d-46ae-b785-71c5ea269ef4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:30.903903 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68dc63-1a1d-46ae-b785-71c5ea269ef4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:30.984746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ed19f-0e1c-4396-875d-9ee99bb90a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:30.986720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ed19f-0e1c-4396-875d-9ee99bb90a8c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:30.987438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ed19f-0e1c-4396-875d-9ee99bb90a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:30.988121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ed19f-0e1c-4396-875d-9ee99bb90a8c None admin] PATH_INFO: `/v3/users/b8be4e8c78444c25a8bd96f5851b92a1` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:30.990935 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ed19f-0e1c-4396-875d-9ee99bb90a8c None admin] RBAC: Authorizing `identity:get_user(user_id=b8be4e8c78444c25a8bd96f5851b92a1)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:30.998152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ed19f-0e1c-4396-875d-9ee99bb90a8c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:31.002360 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 72/135] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:30 2022] GET /identity/v3/users/b8be4e8c78444c25a8bd96f5851b92a1 => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:31.010940 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a3f69-c299-429b-9422-fd325f0f87d9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:31.063380 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:31.233044 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a3f69-c299-429b-9422-fd325f0f87d9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:31.296066 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c18e02-4f4b-4e8c-8832-9cc99e03fa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:31.297619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c18e02-4f4b-4e8c-8832-9cc99e03fa10 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:31.298121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c18e02-4f4b-4e8c-8832-9cc99e03fa10 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:31.298618 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c18e02-4f4b-4e8c-8832-9cc99e03fa10 None admin] PATH_INFO: `/v3/projects/0cf91e07189443cfb23d8c72d10a1e44` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:31.301021 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c18e02-4f4b-4e8c-8832-9cc99e03fa10 None admin] RBAC: Authorizing `identity:get_project(project_id=0cf91e07189443cfb23d8c72d10a1e44)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:31.303822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c18e02-4f4b-4e8c-8832-9cc99e03fa10 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:31.307345 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 64/136] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:31 2022] GET /identity/v3/projects/0cf91e07189443cfb23d8c72d10a1e44 => generated 312 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:31.316382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ed19f-0e1c-4396-875d-9ee99bb90a8c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:31.433773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:31.519629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ed19f-0e1c-4396-875d-9ee99bb90a8c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:31.610348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542f25a-73c8-4bcf-a8e6-c48663812d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:31.612028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542f25a-73c8-4bcf-a8e6-c48663812d24 None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:31.612597 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542f25a-73c8-4bcf-a8e6-c48663812d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:31.613113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542f25a-73c8-4bcf-a8e6-c48663812d24 None admin] PATH_INFO: `/v3/projects/0cf91e07189443cfb23d8c72d10a1e44/users/b8be4e8c78444c25a8bd96f5851b92a1/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:31.617022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542f25a-73c8-4bcf-a8e6-c48663812d24 None admin] RBAC: Authorizing `identity:create_grant(project_id=0cf91e07189443cfb23d8c72d10a1e44, user_id=b8be4e8c78444c25a8bd96f5851b92a1, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:31.619910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542f25a-73c8-4bcf-a8e6-c48663812d24 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:31.653944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 73/137] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:15:31 2022] PUT /identity/v3/projects/0cf91e07189443cfb23d8c72d10a1e44/users/b8be4e8c78444c25a8bd96f5851b92a1/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:15:32.947025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed43167-adfb-4aa0-a6bb-dc48c4102c91 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:32.947572 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed43167-adfb-4aa0-a6bb-dc48c4102c91 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:32.947986 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed43167-adfb-4aa0-a6bb-dc48c4102c91 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:32.950383 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 65/138] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:15:32.960162 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35684361-6a08-47ea-97ab-90a2a45436f7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:32.962722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35684361-6a08-47ea-97ab-90a2a45436f7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:32.963337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35684361-6a08-47ea-97ab-90a2a45436f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:33.006870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-35684361-6a08-47ea-97ab-90a2a45436f7 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:33.008874 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-35684361-6a08-47ea-97ab-90a2a45436f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:33.161801 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 74/139] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:33.224756 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f458b2-118e-4114-822a-3d355a92e644 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:33.225178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f458b2-118e-4114-822a-3d355a92e644 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:33.225706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f458b2-118e-4114-822a-3d355a92e644 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:33.228389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 66/140] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:33.230272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35684361-6a08-47ea-97ab-90a2a45436f7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:33.242942 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9c3fd-2f4b-4ecd-9911-2f1d3ec45a4e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:33.243461 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9c3fd-2f4b-4ecd-9911-2f1d3ec45a4e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:33.244251 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9c3fd-2f4b-4ecd-9911-2f1d3ec45a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:33.313483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-88c9c3fd-2f4b-4ecd-9911-2f1d3ec45a4e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:33.321069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-88c9c3fd-2f4b-4ecd-9911-2f1d3ec45a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:33.474882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:33.619832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 67/141] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:33.658807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35684361-6a08-47ea-97ab-90a2a45436f7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:33.705313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9c3fd-2f4b-4ecd-9911-2f1d3ec45a4e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:33.773089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfa2c5-c9d9-4ad9-a7ee-8dfb5c482f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:33.774374 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfa2c5-c9d9-4ad9-a7ee-8dfb5c482f99 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:33.774865 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfa2c5-c9d9-4ad9-a7ee-8dfb5c482f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:33.775357 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfa2c5-c9d9-4ad9-a7ee-8dfb5c482f99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:33.781034 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dfa2c5-c9d9-4ad9-a7ee-8dfb5c482f99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:33.783738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dfa2c5-c9d9-4ad9-a7ee-8dfb5c482f99 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:33.788916 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 75/142] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:15:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 560 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:33.794985 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:33.804461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfa2c5-c9d9-4ad9-a7ee-8dfb5c482f99 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:33.992159 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9c3fd-2f4b-4ecd-9911-2f1d3ec45a4e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:34.007911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfa2c5-c9d9-4ad9-a7ee-8dfb5c482f99 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:34.059720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:34.074710 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205ecea-5aad-4339-91be-192299f6eef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:34.076201 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205ecea-5aad-4339-91be-192299f6eef2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:34.076710 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205ecea-5aad-4339-91be-192299f6eef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:34.077992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205ecea-5aad-4339-91be-192299f6eef2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:34.100766 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205ecea-5aad-4339-91be-192299f6eef2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:34.102860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e1477-4849-40a1-a874-8fb7a05c32f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:34.103576 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e1477-4849-40a1-a874-8fb7a05c32f6 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:34.103865 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e1477-4849-40a1-a874-8fb7a05c32f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:34.104149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e1477-4849-40a1-a874-8fb7a05c32f6 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:34.104904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e1477-4849-40a1-a874-8fb7a05c32f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:34.106474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e1477-4849-40a1-a874-8fb7a05c32f6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:34.117960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-7205ecea-5aad-4339-91be-192299f6eef2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:15:34.117960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:34.117960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:34.117960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:34.117960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:34.117960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:34.117960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:34.117960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:34.117960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:34.117960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:34.117960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:34.117960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:34.117960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:34.117960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:34.117960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:34.117960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:34.118903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:34.118903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:34.118903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:34.118903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:34.118903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:34.118903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:34.118903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:34.118903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:34.118903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:34.118903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:34.118903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:34.118903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:15:34.118903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:34.118903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:34.118903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:34.118903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:34.119681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:34.119681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:34.119681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:34.119681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:34.119681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:34.119681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:34.119681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:34.119681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:34.119681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:34.119681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:34.119681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:15:34.119681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:15:34.120750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 68/143] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:34.129265 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205ecea-5aad-4339-91be-192299f6eef2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:34.152006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 76/144] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:15:33 2022] POST /identity/v3/users => generated 283 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:15:34.312386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:34.354287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205ecea-5aad-4339-91be-192299f6eef2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:34.427985 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5127b-47f3-46e7-96d9-bc1b8346417b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:34.429139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5127b-47f3-46e7-96d9-bc1b8346417b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:34.429677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5127b-47f3-46e7-96d9-bc1b8346417b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:34.430180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5127b-47f3-46e7-96d9-bc1b8346417b None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:34.431662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5127b-47f3-46e7-96d9-bc1b8346417b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:34.432225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5127b-47f3-46e7-96d9-bc1b8346417b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:34.435015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5127b-47f3-46e7-96d9-bc1b8346417b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:34.437906 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b63d08-9b95-4478-8f05-ebc33e6dcfb9 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:34.438658 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b63d08-9b95-4478-8f05-ebc33e6dcfb9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:34.439400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b63d08-9b95-4478-8f05-ebc33e6dcfb9 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:34.449354 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 77/145] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:34.452026 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 69/146] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:15:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:34.453697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5776d9-2c0f-42c7-b915-ce84985b288c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:34.454606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5776d9-2c0f-42c7-b915-ce84985b288c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:34.455380 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5776d9-2c0f-42c7-b915-ce84985b288c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:34.462123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5127b-47f3-46e7-96d9-bc1b8346417b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:34.503040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-7d5776d9-2c0f-42c7-b915-ce84985b288c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:34.505580 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7d5776d9-2c0f-42c7-b915-ce84985b288c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:34.739188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5127b-47f3-46e7-96d9-bc1b8346417b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:34.755070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 78/147] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:34.827881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5776d9-2c0f-42c7-b915-ce84985b288c None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:34.830321 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419e459-d707-4032-a681-3f38b0a9dc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:34.832243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419e459-d707-4032-a681-3f38b0a9dc5f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:34.833076 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419e459-d707-4032-a681-3f38b0a9dc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:34.833863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419e459-d707-4032-a681-3f38b0a9dc5f None admin] PATH_INFO: `/v3/users/ff214e62139649ca9b5b266907aa8e2c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:34.836439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419e459-d707-4032-a681-3f38b0a9dc5f None admin] RBAC: Authorizing `identity:get_user(user_id=ff214e62139649ca9b5b266907aa8e2c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:34.839595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419e459-d707-4032-a681-3f38b0a9dc5f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:34.844428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 70/148] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:34 2022] GET /identity/v3/users/ff214e62139649ca9b5b266907aa8e2c => generated 289 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:34.855581 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419e459-d707-4032-a681-3f38b0a9dc5f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:34.897674 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:35.053668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419e459-d707-4032-a681-3f38b0a9dc5f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:35.055894 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5776d9-2c0f-42c7-b915-ce84985b288c None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:35.070239 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:35.110189 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb4319-8b55-4bd1-8ff3-f4341d4db5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:35.111027 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb4319-8b55-4bd1-8ff3-f4341d4db5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:35.111316 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb4319-8b55-4bd1-8ff3-f4341d4db5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:35.111595 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb4319-8b55-4bd1-8ff3-f4341d4db5d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:35.112201 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9b7de-968d-479a-be55-1cfb52b63df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:35.113348 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9b7de-968d-479a-be55-1cfb52b63df3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:35.113929 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9b7de-968d-479a-be55-1cfb52b63df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:35.114434 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9b7de-968d-479a-be55-1cfb52b63df3 None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:35.116044 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9b7de-968d-479a-be55-1cfb52b63df3 None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:35.117848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9b7de-968d-479a-be55-1cfb52b63df3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:35.120105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 71/149] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:34 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:35.121898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8519fd-f458-42c0-90e1-aa4112fc6d7d None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:35.122406 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8519fd-f458-42c0-90e1-aa4112fc6d7d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:35.122885 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8519fd-f458-42c0-90e1-aa4112fc6d7d None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:35.124864 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 72/150] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:35.130103 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb4319-8b55-4bd1-8ff3-f4341d4db5d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:35.130675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8519fd-f458-42c0-90e1-aa4112fc6d7d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:35.140071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-80bb4319-8b55-4bd1-8ff3-f4341d4db5d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:15:35.140071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:35.140071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:35.140071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:35.140071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:35.140071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:35.140071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:35.140071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:35.140071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:35.140071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:35.140071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:35.140071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:35.140071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:35.140071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:35.140071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:35.140071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:35.141484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:35.141484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:35.141484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:35.141484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:35.141484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:35.141484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:35.141484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:35.141484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:35.141484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:35.141484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:35.141484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:35.141484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:15:35.141484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:35.141484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:35.141484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:35.141484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:35.142446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:35.142446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:35.142446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:35.142446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:35.142446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:35.142446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:35.142446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:35.142446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:35.142446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:35.142446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:35.142446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:15:35.142446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:15:35.142446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 79/151] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:15:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:15:35.144259 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da3b55-cbfb-4db5-8ec6-c4138d80abbc None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:35.144765 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da3b55-cbfb-4db5-8ec6-c4138d80abbc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:35.146868 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da3b55-cbfb-4db5-8ec6-c4138d80abbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:35.168513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-05da3b55-cbfb-4db5-8ec6-c4138d80abbc None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:35.170497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-05da3b55-cbfb-4db5-8ec6-c4138d80abbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:35.269763 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:35.281774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 80/152] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:35.283502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da3b55-cbfb-4db5-8ec6-c4138d80abbc None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:35.292353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8519fd-f458-42c0-90e1-aa4112fc6d7d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:35.361164 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a08731-26ff-404a-84de-6ff50cc6576c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:35.362135 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a08731-26ff-404a-84de-6ff50cc6576c None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:35.364584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a08731-26ff-404a-84de-6ff50cc6576c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:35.364829 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a08731-26ff-404a-84de-6ff50cc6576c None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582/users/ff214e62139649ca9b5b266907aa8e2c/roles/f4272c5247654dfdbd6abccbd56374ff` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:35.367275 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a08731-26ff-404a-84de-6ff50cc6576c None admin] RBAC: Authorizing `identity:create_grant(project_id=fda8b64de40a4fd698a2458618d1a582, user_id=ff214e62139649ca9b5b266907aa8e2c, role_id=f4272c5247654dfdbd6abccbd56374ff)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:35.368806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a08731-26ff-404a-84de-6ff50cc6576c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:35.388149 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 73/153] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:15:35 2022] PUT /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582/users/ff214e62139649ca9b5b266907aa8e2c/roles/f4272c5247654dfdbd6abccbd56374ff => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:15:35.394939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e82b48-9317-492c-8d48-97587198d266 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:35.396863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e82b48-9317-492c-8d48-97587198d266 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:35.398463 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e82b48-9317-492c-8d48-97587198d266 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:35.403135 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 74/154] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:35 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:35.404335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e82b48-9317-492c-8d48-97587198d266 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:35.506278 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:35.518814 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da3b55-cbfb-4db5-8ec6-c4138d80abbc None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:35.583384 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e82b48-9317-492c-8d48-97587198d266 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:35.625795 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405d1cc-3f87-409e-a29b-1b51aecd3635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:35.626988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405d1cc-3f87-409e-a29b-1b51aecd3635 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:35.627498 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405d1cc-3f87-409e-a29b-1b51aecd3635 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:35.628017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405d1cc-3f87-409e-a29b-1b51aecd3635 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:35.644066 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:35.650097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405d1cc-3f87-409e-a29b-1b51aecd3635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:35.659056 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67407a-b840-489a-a521-a44742a3a67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:35.659734 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67407a-b840-489a-a521-a44742a3a67f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:35.660024 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67407a-b840-489a-a521-a44742a3a67f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:35.660303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67407a-b840-489a-a521-a44742a3a67f None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:35.661087 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67407a-b840-489a-a521-a44742a3a67f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:35.661493 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67407a-b840-489a-a521-a44742a3a67f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:35.662885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67407a-b840-489a-a521-a44742a3a67f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:35.666202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-7405d1cc-3f87-409e-a29b-1b51aecd3635 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:35.666202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:35.666202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:35.666202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:35.666202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:35.666202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:35.666202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:35.666202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:35.666202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:35.666202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:35.666202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:35.666202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:35.666202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:35.666202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:35.666202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:35.666202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:35.667303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:35.667303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:35.667303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:35.667303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:35.667303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:35.667303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:35.667303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:35.667303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:35.667303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:35.667303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:35.667303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:35.667303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:15:35.667303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:35.667303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:35.667303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:35.667303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:35.668611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:35.668611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:35.668611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:35.668611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:35.668611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:35.668611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:35.668611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:35.668611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:35.668611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:35.668611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:35.668611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:35.668611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:15:35.668611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 75/155] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:15:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:35.670187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b426c-114d-40a8-8f97-06cee884abba None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:35.670187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b426c-114d-40a8-8f97-06cee884abba None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:35.670187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b426c-114d-40a8-8f97-06cee884abba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:35.685520 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 81/156] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:15:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:35.686551 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67407a-b840-489a-a521-a44742a3a67f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:35.706464 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-ef2b426c-114d-40a8-8f97-06cee884abba None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:35.708092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ef2b426c-114d-40a8-8f97-06cee884abba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:35.835251 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 76/157] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:35.844025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b426c-114d-40a8-8f97-06cee884abba None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:35.876747 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67407a-b840-489a-a521-a44742a3a67f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:35.925772 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54d4cd-3810-456b-a700-3e3627a59616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:35.926399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54d4cd-3810-456b-a700-3e3627a59616 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:35.926693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54d4cd-3810-456b-a700-3e3627a59616 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:35.926978 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54d4cd-3810-456b-a700-3e3627a59616 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:35.927790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54d4cd-3810-456b-a700-3e3627a59616 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:35.928106 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54d4cd-3810-456b-a700-3e3627a59616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:35.930797 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54d4cd-3810-456b-a700-3e3627a59616 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:35.942581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 82/158] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:35.942792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54d4cd-3810-456b-a700-3e3627a59616 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:35.974102 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:35.982935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b426c-114d-40a8-8f97-06cee884abba None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:36.035362 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:36.056342 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208534c-a73a-43cf-8b91-a8ddc7a0a94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:36.059320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208534c-a73a-43cf-8b91-a8ddc7a0a94a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:36.060278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208534c-a73a-43cf-8b91-a8ddc7a0a94a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:36.061045 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208534c-a73a-43cf-8b91-a8ddc7a0a94a None admin] PATH_INFO: `/v3/users/b8be4e8c78444c25a8bd96f5851b92a1` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:36.064147 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208534c-a73a-43cf-8b91-a8ddc7a0a94a None admin] RBAC: Authorizing `identity:get_user(user_id=b8be4e8c78444c25a8bd96f5851b92a1)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:36.067640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208534c-a73a-43cf-8b91-a8ddc7a0a94a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:36.070489 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54d4cd-3810-456b-a700-3e3627a59616 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:36.074061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 77/159] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:35 2022] GET /identity/v3/users/b8be4e8c78444c25a8bd96f5851b92a1 => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:36.075944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208534c-a73a-43cf-8b91-a8ddc7a0a94a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:36.114713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d90197-3c6e-422b-aa5e-08fcd0c2f269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:36.115860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d90197-3c6e-422b-aa5e-08fcd0c2f269 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:36.116297 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d90197-3c6e-422b-aa5e-08fcd0c2f269 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:36.116721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d90197-3c6e-422b-aa5e-08fcd0c2f269 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:36.135002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d90197-3c6e-422b-aa5e-08fcd0c2f269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:36.149587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-c7d90197-3c6e-422b-aa5e-08fcd0c2f269 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:36.149587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:36.149587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:36.149587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:36.149587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:36.149587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:36.149587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:36.149587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:36.149587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:36.149587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:36.149587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:36.149587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:36.149587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:36.149587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:36.149587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:36.149587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:36.151664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:36.151664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:36.151664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:36.151664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:36.151664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:36.151664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:36.151664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:36.151664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:36.151664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:36.151664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:36.151664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:36.151664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:15:36.151664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:36.151664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:36.151664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:36.151664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:36.153786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:36.153786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:36.153786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:36.153786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:36.153786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:36.153786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:36.153786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:36.153786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:36.153786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:36.153786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:36.153786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:36.153786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:15:36.153786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 83/160] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:36.153786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d90197-3c6e-422b-aa5e-08fcd0c2f269 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:36.259605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:36.282947 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d90197-3c6e-422b-aa5e-08fcd0c2f269 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:36.306195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208534c-a73a-43cf-8b91-a8ddc7a0a94a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:36.324988 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12bc92-edbc-490a-a084-8dc11d102f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:36.325718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12bc92-edbc-490a-a084-8dc11d102f17 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:36.326008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12bc92-edbc-490a-a084-8dc11d102f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:36.326293 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12bc92-edbc-490a-a084-8dc11d102f17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:36.404229 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99103d4-1588-49c3-a596-d9efa4dcf4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:36.406748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99103d4-1588-49c3-a596-d9efa4dcf4ce None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:36.407679 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99103d4-1588-49c3-a596-d9efa4dcf4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:36.408405 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99103d4-1588-49c3-a596-d9efa4dcf4ce None admin] PATH_INFO: `/v3/users/5a11fde32c844e228722284109f82f80` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:36.411087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99103d4-1588-49c3-a596-d9efa4dcf4ce None admin] RBAC: Authorizing `identity:get_user(user_id=5a11fde32c844e228722284109f82f80)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:36.418332 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f12bc92-edbc-490a-a084-8dc11d102f17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:36.419958 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f12bc92-edbc-490a-a084-8dc11d102f17 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:36.421799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99103d4-1588-49c3-a596-d9efa4dcf4ce None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:36.426595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 78/161] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:36 2022] GET /identity/v3/users/5a11fde32c844e228722284109f82f80 => generated 247 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:36.427342 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99103d4-1588-49c3-a596-d9efa4dcf4ce None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:36.549179 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:36.569795 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:36.587478 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 84/162] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:15:36.594638 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c8801-747b-4fd7-8480-38d19f83d42f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:36.596532 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c8801-747b-4fd7-8480-38d19f83d42f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:36.598011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c8801-747b-4fd7-8480-38d19f83d42f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:36.605568 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 85/163] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:36 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:36.606890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c8801-747b-4fd7-8480-38d19f83d42f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:36.649132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99103d4-1588-49c3-a596-d9efa4dcf4ce None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:36.742920 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af98f00-5278-42eb-953e-ef48e90d1ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:36.743541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af98f00-5278-42eb-953e-ef48e90d1ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:36.743974 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af98f00-5278-42eb-953e-ef48e90d1ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:36.744323 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af98f00-5278-42eb-953e-ef48e90d1ff2 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:36.745106 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af98f00-5278-42eb-953e-ef48e90d1ff2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:36.746502 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af98f00-5278-42eb-953e-ef48e90d1ff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:36.747992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af98f00-5278-42eb-953e-ef48e90d1ff2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:36.774804 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 79/164] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:15:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:36.775738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af98f00-5278-42eb-953e-ef48e90d1ff2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:36.892086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c8801-747b-4fd7-8480-38d19f83d42f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:36.975041 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af98f00-5278-42eb-953e-ef48e90d1ff2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:37.040056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785c3b5-8df1-4fef-a2bb-463b0c7f449e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:37.041447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785c3b5-8df1-4fef-a2bb-463b0c7f449e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:37.041922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785c3b5-8df1-4fef-a2bb-463b0c7f449e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:37.042386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785c3b5-8df1-4fef-a2bb-463b0c7f449e None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:37.044754 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7af16e-285b-4487-b3de-9d2c1a3c05ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:37.044985 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785c3b5-8df1-4fef-a2bb-463b0c7f449e None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:37.046701 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7af16e-285b-4487-b3de-9d2c1a3c05ba None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:37.047593 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7af16e-285b-4487-b3de-9d2c1a3c05ba None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:37.048121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7af16e-285b-4487-b3de-9d2c1a3c05ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:37.048238 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785c3b5-8df1-4fef-a2bb-463b0c7f449e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:37.053672 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:37.057073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 80/165] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:36 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:37.059439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b868a4-9383-4766-9d73-a81555bca2dd None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:37.059980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b868a4-9383-4766-9d73-a81555bca2dd None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:37.060469 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b868a4-9383-4766-9d73-a81555bca2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:37.094412 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-55b868a4-9383-4766-9d73-a81555bca2dd None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:37.097248 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-55b868a4-9383-4766-9d73-a81555bca2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:37.240866 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 81/166] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:37.242579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b868a4-9383-4766-9d73-a81555bca2dd None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:37.279571 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:37.400966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7af16e-285b-4487-b3de-9d2c1a3c05ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:37.403741 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7af16e-285b-4487-b3de-9d2c1a3c05ba None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:37.596808 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b868a4-9383-4766-9d73-a81555bca2dd None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:37.620276 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 86/167] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1014 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:15:37.621375 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7af16e-285b-4487-b3de-9d2c1a3c05ba None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:37.697738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:37.720267 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe2dd7-f273-4825-a8be-eb9130aa6288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:37.722369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe2dd7-f273-4825-a8be-eb9130aa6288 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:37.723177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe2dd7-f273-4825-a8be-eb9130aa6288 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:37.724279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe2dd7-f273-4825-a8be-eb9130aa6288 None admin] PATH_INFO: `/v3/users/b8be4e8c78444c25a8bd96f5851b92a1` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:37.742684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe2dd7-f273-4825-a8be-eb9130aa6288 None admin] RBAC: Authorizing `identity:get_user(user_id=b8be4e8c78444c25a8bd96f5851b92a1)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:37.754250 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe2dd7-f273-4825-a8be-eb9130aa6288 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:37.762102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 82/168] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:37 2022] GET /identity/v3/users/b8be4e8c78444c25a8bd96f5851b92a1 => generated 275 bytes in 521 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:37.771303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe2dd7-f273-4825-a8be-eb9130aa6288 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:37.867959 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:37.892678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7af16e-285b-4487-b3de-9d2c1a3c05ba None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:37.966374 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ea6dc-4200-4c9b-a7d1-702c02590bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:37.972513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ea6dc-4200-4c9b-a7d1-702c02590bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:37.972513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ea6dc-4200-4c9b-a7d1-702c02590bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:37.972513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ea6dc-4200-4c9b-a7d1-702c02590bb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:37.972513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ea6dc-4200-4c9b-a7d1-702c02590bb1 None admin] RBAC: Adding query filter params (role.id=6e5dce7d61b34704ad2cd32251fa4d34, scope.project.id=fda8b64de40a4fd698a2458618d1a582, user.id=b8be4e8c78444c25a8bd96f5851b92a1) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:37.972513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ea6dc-4200-4c9b-a7d1-702c02590bb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:37.972513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ea6dc-4200-4c9b-a7d1-702c02590bb1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:37.986453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 87/169] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:15:37 2022] GET /identity/v3/role_assignments?user.id=b8be4e8c78444c25a8bd96f5851b92a1&scope.project.id=fda8b64de40a4fd698a2458618d1a582&role.id=6e5dce7d61b34704ad2cd32251fa4d34 => generated 608 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:37.986453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ea6dc-4200-4c9b-a7d1-702c02590bb1 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:38.166850 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe2dd7-f273-4825-a8be-eb9130aa6288 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:38.174649 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ea6dc-4200-4c9b-a7d1-702c02590bb1 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:38.243223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da19ea-0fba-49c1-aa98-fbcfec98b439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:38.243223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da19ea-0fba-49c1-aa98-fbcfec98b439 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:38.243223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da19ea-0fba-49c1-aa98-fbcfec98b439 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:38.243223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da19ea-0fba-49c1-aa98-fbcfec98b439 None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:38.246535 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da19ea-0fba-49c1-aa98-fbcfec98b439 None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:38.250073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da19ea-0fba-49c1-aa98-fbcfec98b439 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:38.253168 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 88/170] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:37 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:38.253168 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da19ea-0fba-49c1-aa98-fbcfec98b439 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:38.286756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:38.299422 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c5085-11a0-4d61-b76e-6e0500e8bcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:38.300740 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c5085-11a0-4d61-b76e-6e0500e8bcb0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:38.301220 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c5085-11a0-4d61-b76e-6e0500e8bcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:38.301997 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c5085-11a0-4d61-b76e-6e0500e8bcb0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:38.345149 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c5085-11a0-4d61-b76e-6e0500e8bcb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:38.353186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-256c5085-11a0-4d61-b76e-6e0500e8bcb0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:38.353186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:38.353186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:38.353186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:38.353186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:38.353186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:38.353186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:38.353186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:38.353186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:38.353186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:38.353186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:38.353186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:38.353186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:38.353186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:38.353186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:38.353186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:38.356364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:38.356364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:38.356364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:38.356364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:38.356364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:38.356364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:38.356364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:38.356364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:38.356364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:38.356364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:38.356364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:38.356364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:15:38.356364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:38.356364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:38.356364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:38.356364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:38.373079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:38.373079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:38.373079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:38.373079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:38.373079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:38.373079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:38.373079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:38.373079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:38.373079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:38.373079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:38.373079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:38.373079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:15:38.373079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 83/171] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 589 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:38.373079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c5085-11a0-4d61-b76e-6e0500e8bcb0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:38.541949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:38.589479 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da19ea-0fba-49c1-aa98-fbcfec98b439 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:38.667924 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c9e00-292c-46cc-9f31-a01f35155ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:38.669757 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c9e00-292c-46cc-9f31-a01f35155ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:38.671156 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c9e00-292c-46cc-9f31-a01f35155ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:38.671972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c9e00-292c-46cc-9f31-a01f35155ed3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:38.683554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c5085-11a0-4d61-b76e-6e0500e8bcb0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:38.772673 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8cf106-b944-43f7-bf30-09f19a030834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:38.774450 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cf106-b944-43f7-bf30-09f19a030834 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:38.775242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cf106-b944-43f7-bf30-09f19a030834 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:38.776016 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cf106-b944-43f7-bf30-09f19a030834 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:38.777994 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8cf106-b944-43f7-bf30-09f19a030834 None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=fda8b64de40a4fd698a2458618d1a582, user.id=5a11fde32c844e228722284109f82f80) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:38.778899 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8cf106-b944-43f7-bf30-09f19a030834 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:38.783035 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8cf106-b944-43f7-bf30-09f19a030834 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:38.812328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 84/172] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:15:38 2022] GET /identity/v3/role_assignments?user.id=5a11fde32c844e228722284109f82f80&scope.project.id=fda8b64de40a4fd698a2458618d1a582&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 608 bytes in 457 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:38.813591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8cf106-b944-43f7-bf30-09f19a030834 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:38.818260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c9e00-292c-46cc-9f31-a01f35155ed3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:38.819839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c9e00-292c-46cc-9f31-a01f35155ed3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:38.861015 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:39.024026 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8cf106-b944-43f7-bf30-09f19a030834 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:39.058375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:39.084949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 89/173] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 832 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:15:39.093916 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c9e00-292c-46cc-9f31-a01f35155ed3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:39.095735 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca77d3-6253-4a4c-9400-e5fe468454ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:39.096810 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca77d3-6253-4a4c-9400-e5fe468454ae None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:39.097230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca77d3-6253-4a4c-9400-e5fe468454ae None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:39.097658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca77d3-6253-4a4c-9400-e5fe468454ae None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:39.098840 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca77d3-6253-4a4c-9400-e5fe468454ae None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:39.099403 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca77d3-6253-4a4c-9400-e5fe468454ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:39.102200 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca77d3-6253-4a4c-9400-e5fe468454ae None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:39.120694 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 85/174] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:15:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:39.130437 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca77d3-6253-4a4c-9400-e5fe468454ae None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:39.267433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c9e00-292c-46cc-9f31-a01f35155ed3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:39.284133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:39.308826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ecc9b1-92ca-492f-8740-a403d35aa08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:39.309777 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecc9b1-92ca-492f-8740-a403d35aa08e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:39.310049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecc9b1-92ca-492f-8740-a403d35aa08e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:39.310322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecc9b1-92ca-492f-8740-a403d35aa08e None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:39.311958 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ecc9b1-92ca-492f-8740-a403d35aa08e None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:39.313535 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ecc9b1-92ca-492f-8740-a403d35aa08e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:39.315686 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 90/175] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:39 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:39.323134 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ecc9b1-92ca-492f-8740-a403d35aa08e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:39.499760 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ecc9b1-92ca-492f-8740-a403d35aa08e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:39.512225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca77d3-6253-4a4c-9400-e5fe468454ae None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:39.604811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38feea3-ba9a-4a28-9f3c-da45a2569529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:39.605914 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38feea3-ba9a-4a28-9f3c-da45a2569529 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:39.606512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38feea3-ba9a-4a28-9f3c-da45a2569529 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:39.606927 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38feea3-ba9a-4a28-9f3c-da45a2569529 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:39.614197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38feea3-ba9a-4a28-9f3c-da45a2569529 None admin] RBAC: Adding query filter params (role.id=ed6cf002b70f422190946ebcb114da60, scope.project.id=fda8b64de40a4fd698a2458618d1a582, user.id=b8be4e8c78444c25a8bd96f5851b92a1) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:39.614723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38feea3-ba9a-4a28-9f3c-da45a2569529 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:39.617657 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38feea3-ba9a-4a28-9f3c-da45a2569529 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:39.641887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 91/176] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:15:39 2022] GET /identity/v3/role_assignments?user.id=b8be4e8c78444c25a8bd96f5851b92a1&scope.project.id=fda8b64de40a4fd698a2458618d1a582&role.id=ed6cf002b70f422190946ebcb114da60 => generated 608 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:39.664357 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ff65d-6b18-405b-9c3d-507f6549edf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:39.665996 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ff65d-6b18-405b-9c3d-507f6549edf0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:39.666513 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ff65d-6b18-405b-9c3d-507f6549edf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:39.667019 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ff65d-6b18-405b-9c3d-507f6549edf0 None admin] PATH_INFO: `/v3/users/b8be4e8c78444c25a8bd96f5851b92a1` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:39.673597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ff65d-6b18-405b-9c3d-507f6549edf0 None admin] RBAC: Authorizing `identity:get_user(user_id=b8be4e8c78444c25a8bd96f5851b92a1)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:39.676759 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ff65d-6b18-405b-9c3d-507f6549edf0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:39.680816 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 86/177] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:39 2022] GET /identity/v3/users/b8be4e8c78444c25a8bd96f5851b92a1 => generated 275 bytes in 551 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:39.687588 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38feea3-ba9a-4a28-9f3c-da45a2569529 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:39.782169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:39.806446 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2f585-2a35-4b5f-bbbb-961b24f780a1 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:39.807386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2f585-2a35-4b5f-bbbb-961b24f780a1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:39.808160 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2f585-2a35-4b5f-bbbb-961b24f780a1 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:39.814486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 87/178] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:39.824654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bb731-841c-43a5-8e8b-37134c165c33 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:39.825833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bb731-841c-43a5-8e8b-37134c165c33 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:39.826733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bb731-841c-43a5-8e8b-37134c165c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:39.849541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38feea3-ba9a-4a28-9f3c-da45a2569529 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:39.864925 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e11bb731-841c-43a5-8e8b-37134c165c33 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:39.867575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e11bb731-841c-43a5-8e8b-37134c165c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:39.898443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b73610-03b2-4a7e-aec0-1ee74f40b0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:39.899120 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b73610-03b2-4a7e-aec0-1ee74f40b0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:39.899397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b73610-03b2-4a7e-aec0-1ee74f40b0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:39.899677 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b73610-03b2-4a7e-aec0-1ee74f40b0c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:39.989748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:40.038826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b73610-03b2-4a7e-aec0-1ee74f40b0c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:40.041561 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b73610-03b2-4a7e-aec0-1ee74f40b0c5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:40.052214 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 88/179] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:40.123978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11bb731-841c-43a5-8e8b-37134c165c33 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:40.159678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:40.317423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 92/180] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:15:40.319601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fc269-f9f3-46da-907c-d3e311d30306 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:40.320173 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fc269-f9f3-46da-907c-d3e311d30306 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:40.320679 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fc269-f9f3-46da-907c-d3e311d30306 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:40.323154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 93/181] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:40.324770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fc269-f9f3-46da-907c-d3e311d30306 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:40.366375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11bb731-841c-43a5-8e8b-37134c165c33 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:40.445144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea63757-3491-4ff2-a944-6f67e827bcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:40.448383 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea63757-3491-4ff2-a944-6f67e827bcf2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:40.449428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea63757-3491-4ff2-a944-6f67e827bcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:40.450185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea63757-3491-4ff2-a944-6f67e827bcf2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:40.478812 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea63757-3491-4ff2-a944-6f67e827bcf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:40.490155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-9ea63757-3491-4ff2-a944-6f67e827bcf2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:15:40.490155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:40.490155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:40.490155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:40.490155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:40.490155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:40.490155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:40.490155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:40.490155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:40.490155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:40.490155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:40.490155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:40.490155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:40.490155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:40.490155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:40.490155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:40.491905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:40.491905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:40.491905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:40.491905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:40.491905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:40.491905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:40.491905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:40.491905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:40.491905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:40.491905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:40.491905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:40.491905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:15:40.491905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:40.491905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:40.491905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:40.491905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:40.493471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:40.493471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:40.493471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:40.493471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:40.493471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:40.493471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:40.493471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:40.493471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:40.493471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:40.493471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:40.493471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:15:40.493471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:15:40.493471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 89/182] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:40.493471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9ff0a-38d3-41a8-9c26-b659668346d6 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:40.494823 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9ff0a-38d3-41a8-9c26-b659668346d6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:40.494823 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9ff0a-38d3-41a8-9c26-b659668346d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:40.527410 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fc269-f9f3-46da-907c-d3e311d30306 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:40.532269 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-dcc9ff0a-38d3-41a8-9c26-b659668346d6 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:40.535441 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-dcc9ff0a-38d3-41a8-9c26-b659668346d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:40.583651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ab219-f1ef-40be-b7e7-deffd79ce31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:40.584538 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ab219-f1ef-40be-b7e7-deffd79ce31b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:40.584826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ab219-f1ef-40be-b7e7-deffd79ce31b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:40.585105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ab219-f1ef-40be-b7e7-deffd79ce31b None admin] PATH_INFO: `/v3/projects/0cf91e07189443cfb23d8c72d10a1e44` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:40.586810 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ab219-f1ef-40be-b7e7-deffd79ce31b None admin] RBAC: Authorizing `identity:get_project(project_id=0cf91e07189443cfb23d8c72d10a1e44)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:40.588121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ab219-f1ef-40be-b7e7-deffd79ce31b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:40.589941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 94/183] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:40 2022] GET /identity/v3/projects/0cf91e07189443cfb23d8c72d10a1e44 => generated 312 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:40.590796 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ab219-f1ef-40be-b7e7-deffd79ce31b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:40.603370 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:40.623433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:40.674041 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 90/184] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:40.683864 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9ff0a-38d3-41a8-9c26-b659668346d6 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:40.729234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ab219-f1ef-40be-b7e7-deffd79ce31b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:40.771172 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32434e8e-7337-4df9-bad8-49edfa621d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:40.771877 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32434e8e-7337-4df9-bad8-49edfa621d4f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:40.772163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32434e8e-7337-4df9-bad8-49edfa621d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:40.772446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32434e8e-7337-4df9-bad8-49edfa621d4f None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:40.773259 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32434e8e-7337-4df9-bad8-49edfa621d4f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:40.773595 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32434e8e-7337-4df9-bad8-49edfa621d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:40.775055 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32434e8e-7337-4df9-bad8-49edfa621d4f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:40.786156 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 95/185] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:15:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:40.787382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa434ec-f630-45e5-b43f-21e1ba096319 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:40.787674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa434ec-f630-45e5-b43f-21e1ba096319 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:40.787958 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa434ec-f630-45e5-b43f-21e1ba096319 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:40.789323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 96/186] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:40.794284 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa434ec-f630-45e5-b43f-21e1ba096319 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:40.872457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9ff0a-38d3-41a8-9c26-b659668346d6 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:40.907175 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:40.925201 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e591c9b-c259-42d5-90f1-aa7af5af5475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:40.926411 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e591c9b-c259-42d5-90f1-aa7af5af5475 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:40.927009 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e591c9b-c259-42d5-90f1-aa7af5af5475 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:40.927425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e591c9b-c259-42d5-90f1-aa7af5af5475 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:40.929162 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e591c9b-c259-42d5-90f1-aa7af5af5475 None admin] RBAC: Adding query filter params (role.id=ed6cf002b70f422190946ebcb114da60, scope.project.id=0cf91e07189443cfb23d8c72d10a1e44, user.id=b8be4e8c78444c25a8bd96f5851b92a1) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:40.929750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e591c9b-c259-42d5-90f1-aa7af5af5475 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:40.932593 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e591c9b-c259-42d5-90f1-aa7af5af5475 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:40.938342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa434ec-f630-45e5-b43f-21e1ba096319 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:40.953391 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 91/187] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:15:40 2022] GET /identity/v3/role_assignments?user.id=b8be4e8c78444c25a8bd96f5851b92a1&scope.project.id=0cf91e07189443cfb23d8c72d10a1e44&role.id=ed6cf002b70f422190946ebcb114da60 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:40.953391 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e591c9b-c259-42d5-90f1-aa7af5af5475 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:41.006532 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca40e17-f0cd-4604-9996-6ab1f6b6183e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:41.008006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca40e17-f0cd-4604-9996-6ab1f6b6183e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:41.008614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca40e17-f0cd-4604-9996-6ab1f6b6183e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:41.009152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca40e17-f0cd-4604-9996-6ab1f6b6183e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:41.028285 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:41.031640 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca40e17-f0cd-4604-9996-6ab1f6b6183e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:41.046517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-2ca40e17-f0cd-4604-9996-6ab1f6b6183e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:41.046517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:41.046517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:41.046517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:41.046517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:41.046517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:41.046517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:41.046517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:41.046517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:41.046517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:41.046517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:41.046517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:41.046517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:41.046517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:41.046517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:41.046517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:41.048165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:41.048165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:41.048165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:41.048165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:41.048165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:41.048165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:41.048165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:41.048165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:41.048165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:41.048165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:41.048165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:41.048165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:15:41.048165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:41.048165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:41.048165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:41.048165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:41.049816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:41.049816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:41.049816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:41.049816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:41.049816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:41.049816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:41.049816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:41.049816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:41.049816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:41.049816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:41.049816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:41.049816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:15:41.054541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 97/188] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:15:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:41.054541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787ad18-530b-44a1-a246-2d33e3e4891c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:41.054968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787ad18-530b-44a1-a246-2d33e3e4891c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:41.055066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787ad18-530b-44a1-a246-2d33e3e4891c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:41.083236 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-1787ad18-530b-44a1-a246-2d33e3e4891c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:41.084678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1787ad18-530b-44a1-a246-2d33e3e4891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:41.144409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e591c9b-c259-42d5-90f1-aa7af5af5475 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:41.204001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 98/189] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:41.204412 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32285ac0-b644-4faf-bcd6-11f008884f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:41.205926 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32285ac0-b644-4faf-bcd6-11f008884f3a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:41.206394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32285ac0-b644-4faf-bcd6-11f008884f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:41.206709 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787ad18-530b-44a1-a246-2d33e3e4891c None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:41.206892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32285ac0-b644-4faf-bcd6-11f008884f3a None admin] PATH_INFO: `/v3/users/ff214e62139649ca9b5b266907aa8e2c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:41.210918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32285ac0-b644-4faf-bcd6-11f008884f3a None admin] RBAC: Authorizing `identity:get_user(user_id=ff214e62139649ca9b5b266907aa8e2c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:41.213403 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32285ac0-b644-4faf-bcd6-11f008884f3a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:41.216429 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 92/190] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:40 2022] GET /identity/v3/users/ff214e62139649ca9b5b266907aa8e2c => generated 289 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:41.225509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13219507-00bf-4783-9719-ea56963d14c3 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:41.225509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13219507-00bf-4783-9719-ea56963d14c3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:41.225509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13219507-00bf-4783-9719-ea56963d14c3 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:41.228788 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 93/191] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:41.230019 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13219507-00bf-4783-9719-ea56963d14c3 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:41.312945 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:41.368770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787ad18-530b-44a1-a246-2d33e3e4891c None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:41.378452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:41.387341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13219507-00bf-4783-9719-ea56963d14c3 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:41.412877 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040b855-ea31-44f1-a68e-24e1f10a0909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:41.414016 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040b855-ea31-44f1-a68e-24e1f10a0909 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:41.414532 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040b855-ea31-44f1-a68e-24e1f10a0909 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:41.415037 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040b855-ea31-44f1-a68e-24e1f10a0909 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:41.416509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040b855-ea31-44f1-a68e-24e1f10a0909 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:41.417089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040b855-ea31-44f1-a68e-24e1f10a0909 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:41.419859 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040b855-ea31-44f1-a68e-24e1f10a0909 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:41.434265 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35927d-62be-4036-b08d-342b8cc4adba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:41.435024 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35927d-62be-4036-b08d-342b8cc4adba None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:41.435374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35927d-62be-4036-b08d-342b8cc4adba None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:41.435705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35927d-62be-4036-b08d-342b8cc4adba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:41.435887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 99/192] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:41.438750 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040b855-ea31-44f1-a68e-24e1f10a0909 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:41.537982 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35927d-62be-4036-b08d-342b8cc4adba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:41.540536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35927d-62be-4036-b08d-342b8cc4adba None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:41.577238 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040b855-ea31-44f1-a68e-24e1f10a0909 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:41.612371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:41.625579 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d4f9c-5636-4c98-9fbb-a0d502b3b928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:41.626593 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d4f9c-5636-4c98-9fbb-a0d502b3b928 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:41.626934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d4f9c-5636-4c98-9fbb-a0d502b3b928 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:41.627296 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d4f9c-5636-4c98-9fbb-a0d502b3b928 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:41.629047 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d4f9c-5636-4c98-9fbb-a0d502b3b928 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:41.630830 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d4f9c-5636-4c98-9fbb-a0d502b3b928 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:41.633084 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 100/193] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:15:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:41.634562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbb8ef-d8e2-4f16-933c-415d20e47bb8 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:41.635007 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbb8ef-d8e2-4f16-933c-415d20e47bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:41.635410 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbb8ef-d8e2-4f16-933c-415d20e47bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:41.656575 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-6bfbb8ef-d8e2-4f16-933c-415d20e47bb8 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:41.658121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6bfbb8ef-d8e2-4f16-933c-415d20e47bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:41.683168 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:41.691853 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 94/194] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:15:41.692777 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35927d-62be-4036-b08d-342b8cc4adba None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:41.750767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 101/195] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:41.751624 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbb8ef-d8e2-4f16-933c-415d20e47bb8 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:41.856087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35927d-62be-4036-b08d-342b8cc4adba None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:41.862060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbb8ef-d8e2-4f16-933c-415d20e47bb8 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:41.893660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:41.898995 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c5a52-7cd5-4475-ad4f-a8fbf8bec779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:41.900574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c5a52-7cd5-4475-ad4f-a8fbf8bec779 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:41.901073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c5a52-7cd5-4475-ad4f-a8fbf8bec779 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:41.901605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c5a52-7cd5-4475-ad4f-a8fbf8bec779 None admin] PATH_INFO: `/v3/users/ea9d722d78de4f4fa7c1aceb35b551fc` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:41.903353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f24b5-34f5-46e2-a8fe-982513617286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:41.904066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f24b5-34f5-46e2-a8fe-982513617286 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:41.904398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f24b5-34f5-46e2-a8fe-982513617286 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:41.905667 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f24b5-34f5-46e2-a8fe-982513617286 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:41.906272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f24b5-34f5-46e2-a8fe-982513617286 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:41.907973 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f24b5-34f5-46e2-a8fe-982513617286 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:41.926195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c5a52-7cd5-4475-ad4f-a8fbf8bec779 None admin] RBAC: Authorizing `identity:get_user(user_id=ea9d722d78de4f4fa7c1aceb35b551fc)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:41.928267 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c5a52-7cd5-4475-ad4f-a8fbf8bec779 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:41.930361 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 95/196] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:41 2022] GET /identity/v3/users/ea9d722d78de4f4fa7c1aceb35b551fc => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:41.931093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c5a52-7cd5-4475-ad4f-a8fbf8bec779 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:41.947797 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 102/197] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:15:41 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:15:41.947797 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f24b5-34f5-46e2-a8fe-982513617286 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:42.031369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:42.119739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c5a52-7cd5-4475-ad4f-a8fbf8bec779 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:42.147462 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f24b5-34f5-46e2-a8fe-982513617286 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:42.182375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c5b6e-ef51-436c-93f0-c8068f9f8410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:42.183918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c5b6e-ef51-436c-93f0-c8068f9f8410 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:42.184427 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c5b6e-ef51-436c-93f0-c8068f9f8410 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:42.184957 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c5b6e-ef51-436c-93f0-c8068f9f8410 None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:42.187268 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c5b6e-ef51-436c-93f0-c8068f9f8410 None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:42.190800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c5b6e-ef51-436c-93f0-c8068f9f8410 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:42.193733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 96/198] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:41 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:42.198873 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c5b6e-ef51-436c-93f0-c8068f9f8410 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:42.213593 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616978d2-8ff7-4258-9752-a1c18a7dc0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:42.214877 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616978d2-8ff7-4258-9752-a1c18a7dc0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:42.215359 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616978d2-8ff7-4258-9752-a1c18a7dc0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:42.215826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616978d2-8ff7-4258-9752-a1c18a7dc0e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:42.233862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616978d2-8ff7-4258-9752-a1c18a7dc0e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:42.245986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-616978d2-8ff7-4258-9752-a1c18a7dc0e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:15:42.245986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:42.245986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:42.245986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:42.245986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:42.245986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:42.245986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:42.245986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:42.245986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:42.245986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:42.245986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:42.245986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:42.245986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:42.245986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:42.245986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:42.245986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:42.247731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:42.247731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:42.247731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:42.247731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:42.247731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:42.247731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:42.247731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:42.247731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:42.247731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:42.247731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:42.247731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:42.247731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:15:42.247731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:42.247731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:42.247731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:42.247731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:42.249972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:42.249972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:42.249972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:42.249972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:42.249972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:42.249972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:42.249972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:42.249972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:42.249972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:42.249972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:42.249972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:15:42.249972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:15:42.249972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 103/199] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:15:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:42.254505 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616978d2-8ff7-4258-9752-a1c18a7dc0e8 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:42.338121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:42.370210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c5b6e-ef51-436c-93f0-c8068f9f8410 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:42.430571 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:42.435685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616978d2-8ff7-4258-9752-a1c18a7dc0e8 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:42.463450 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d7369-46ec-4e73-9cb5-2304e216d9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:42.464121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d7369-46ec-4e73-9cb5-2304e216d9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:42.464639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d7369-46ec-4e73-9cb5-2304e216d9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:42.464925 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d7369-46ec-4e73-9cb5-2304e216d9a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:42.500807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb27957-b985-4eea-b5cb-9ba2f65078ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:42.502890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb27957-b985-4eea-b5cb-9ba2f65078ec None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:42.503826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb27957-b985-4eea-b5cb-9ba2f65078ec None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:42.504592 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb27957-b985-4eea-b5cb-9ba2f65078ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:42.506554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb27957-b985-4eea-b5cb-9ba2f65078ec None admin] RBAC: Adding query filter params (role.id=f4272c5247654dfdbd6abccbd56374ff, scope.project.id=fda8b64de40a4fd698a2458618d1a582, user.id=ff214e62139649ca9b5b266907aa8e2c) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:42.507397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb27957-b985-4eea-b5cb-9ba2f65078ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:42.511377 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb27957-b985-4eea-b5cb-9ba2f65078ec None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:42.523846 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 104/200] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:15:42 2022] GET /identity/v3/role_assignments?user.id=ff214e62139649ca9b5b266907aa8e2c&scope.project.id=fda8b64de40a4fd698a2458618d1a582&role.id=f4272c5247654dfdbd6abccbd56374ff => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:42.532170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb27957-b985-4eea-b5cb-9ba2f65078ec None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:42.586483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d7369-46ec-4e73-9cb5-2304e216d9a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:42.593335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d7369-46ec-4e73-9cb5-2304e216d9a5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:42.789983 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 97/201] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:15:42.796740 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d7369-46ec-4e73-9cb5-2304e216d9a5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:42.826452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:42.892247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb27957-b985-4eea-b5cb-9ba2f65078ec None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:42.909063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:42.933448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d7369-46ec-4e73-9cb5-2304e216d9a5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:42.953720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45f92f-3e52-48c4-963c-7a09836d866d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:42.954428 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45f92f-3e52-48c4-963c-7a09836d866d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:42.954715 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45f92f-3e52-48c4-963c-7a09836d866d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:42.955002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45f92f-3e52-48c4-963c-7a09836d866d None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:42.955784 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45f92f-3e52-48c4-963c-7a09836d866d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:42.956094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45f92f-3e52-48c4-963c-7a09836d866d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:42.959041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45f92f-3e52-48c4-963c-7a09836d866d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:42.985308 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 105/202] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:15:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:42.991923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45f92f-3e52-48c4-963c-7a09836d866d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:42.997927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fdc900-d06a-4d72-8c6b-a2d78585a59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:42.997927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdc900-d06a-4d72-8c6b-a2d78585a59b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:42.998307 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdc900-d06a-4d72-8c6b-a2d78585a59b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:42.999199 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdc900-d06a-4d72-8c6b-a2d78585a59b None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:43.001113 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fdc900-d06a-4d72-8c6b-a2d78585a59b None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:43.002850 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fdc900-d06a-4d72-8c6b-a2d78585a59b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:43.009465 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 98/203] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:42 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:43.012196 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fdc900-d06a-4d72-8c6b-a2d78585a59b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:43.149629 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:43.167008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45f92f-3e52-48c4-963c-7a09836d866d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:43.169333 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fdc900-d06a-4d72-8c6b-a2d78585a59b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:43.212040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3d4b4-488a-473b-a1f1-1d47fb2de32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:43.213252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3d4b4-488a-473b-a1f1-1d47fb2de32e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:43.213752 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3d4b4-488a-473b-a1f1-1d47fb2de32e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:43.214983 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3d4b4-488a-473b-a1f1-1d47fb2de32e None admin] PATH_INFO: `/v3/users/ea9d722d78de4f4fa7c1aceb35b551fc` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:43.216655 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3d4b4-488a-473b-a1f1-1d47fb2de32e None admin] RBAC: Authorizing `identity:get_user(user_id=ea9d722d78de4f4fa7c1aceb35b551fc)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:43.219485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3d4b4-488a-473b-a1f1-1d47fb2de32e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:43.222567 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 106/204] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:42 2022] GET /identity/v3/users/ea9d722d78de4f4fa7c1aceb35b551fc => generated 283 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:43.233361 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3d4b4-488a-473b-a1f1-1d47fb2de32e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:43.238327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6f01f-c7fd-4b0a-a647-7d47d4d441f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:43.239006 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6f01f-c7fd-4b0a-a647-7d47d4d441f4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:43.239291 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6f01f-c7fd-4b0a-a647-7d47d4d441f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:43.239568 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6f01f-c7fd-4b0a-a647-7d47d4d441f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:43.240416 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6f01f-c7fd-4b0a-a647-7d47d4d441f4 None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78, user.id=ea9d722d78de4f4fa7c1aceb35b551fc) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:43.240760 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6f01f-c7fd-4b0a-a647-7d47d4d441f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:43.242245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6f01f-c7fd-4b0a-a647-7d47d4d441f4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:43.250288 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 99/205] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:15:43 2022] GET /identity/v3/role_assignments?user.id=ea9d722d78de4f4fa7c1aceb35b551fc&scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:43.302401 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:43.438953 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3d4b4-488a-473b-a1f1-1d47fb2de32e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:43.505746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12ed3b-4f5b-4ad2-9a8b-0c6c2eddff2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:43.506868 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12ed3b-4f5b-4ad2-9a8b-0c6c2eddff2b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:43.507365 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12ed3b-4f5b-4ad2-9a8b-0c6c2eddff2b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:43.507842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12ed3b-4f5b-4ad2-9a8b-0c6c2eddff2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:43.645947 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12ed3b-4f5b-4ad2-9a8b-0c6c2eddff2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:43.648788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12ed3b-4f5b-4ad2-9a8b-0c6c2eddff2b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:43.733962 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:43.829802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 107/206] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 06:15:43.835194 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6f01f-c7fd-4b0a-a647-7d47d4d441f4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:43.951473 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6f01f-c7fd-4b0a-a647-7d47d4d441f4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:43.990643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f739df-8995-4a28-bde4-b4b37ffaeb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:43.991445 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f739df-8995-4a28-bde4-b4b37ffaeb61 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:43.991734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f739df-8995-4a28-bde4-b4b37ffaeb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:43.992010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f739df-8995-4a28-bde4-b4b37ffaeb61 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:43.993327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f739df-8995-4a28-bde4-b4b37ffaeb61 None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:43.994796 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f739df-8995-4a28-bde4-b4b37ffaeb61 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:43.996883 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 100/207] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:43 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:44.003735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12ed3b-4f5b-4ad2-9a8b-0c6c2eddff2b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:44.126875 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12ed3b-4f5b-4ad2-9a8b-0c6c2eddff2b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:44.171615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f202830-c3b8-4c77-8480-92acc24ffa3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:44.172305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f202830-c3b8-4c77-8480-92acc24ffa3c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:44.172588 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f202830-c3b8-4c77-8480-92acc24ffa3c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:44.172862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f202830-c3b8-4c77-8480-92acc24ffa3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:44.173773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f202830-c3b8-4c77-8480-92acc24ffa3c None admin] RBAC: Adding query filter params (role.id=6e5dce7d61b34704ad2cd32251fa4d34, scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78, user.id=ea9d722d78de4f4fa7c1aceb35b551fc) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:44.174089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f202830-c3b8-4c77-8480-92acc24ffa3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:44.175605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f202830-c3b8-4c77-8480-92acc24ffa3c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:44.179152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:44.183043 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 108/208] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:15:44 2022] GET /identity/v3/role_assignments?user.id=ea9d722d78de4f4fa7c1aceb35b551fc&scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78&role.id=6e5dce7d61b34704ad2cd32251fa4d34 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:44.992779 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc3b74-ea74-4995-9431-b0165c301e5b None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:44.993410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc3b74-ea74-4995-9431-b0165c301e5b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:44.993410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc3b74-ea74-4995-9431-b0165c301e5b None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:44.995185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 101/209] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:45.002004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00aff10-29eb-4ff4-aa02-bcad2c68e04c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:45.002801 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00aff10-29eb-4ff4-aa02-bcad2c68e04c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:45.002801 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00aff10-29eb-4ff4-aa02-bcad2c68e04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:45.021762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e00aff10-29eb-4ff4-aa02-bcad2c68e04c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:45.022983 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e00aff10-29eb-4ff4-aa02-bcad2c68e04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:45.098150 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 109/210] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:45.122953 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc3b74-ea74-4995-9431-b0165c301e5b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:45.153358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:45.189128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abedae-6989-44f7-9eda-7e02328f6f4e None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:45.189622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abedae-6989-44f7-9eda-7e02328f6f4e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:45.189960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abedae-6989-44f7-9eda-7e02328f6f4e None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:45.191833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 110/211] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:45.198679 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfa266-4f31-4e4f-9387-ced76dd56d0e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:45.199136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfa266-4f31-4e4f-9387-ced76dd56d0e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:45.199430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfa266-4f31-4e4f-9387-ced76dd56d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:45.216862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-bfdfa266-4f31-4e4f-9387-ced76dd56d0e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:45.218319 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bfdfa266-4f31-4e4f-9387-ced76dd56d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:45.243240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc3b74-ea74-4995-9431-b0165c301e5b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:45.282258 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d2628-ed11-4586-b8a2-47f6b62e88a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:45.283415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d2628-ed11-4586-b8a2-47f6b62e88a8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:45.283903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d2628-ed11-4586-b8a2-47f6b62e88a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:45.284397 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d2628-ed11-4586-b8a2-47f6b62e88a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:45.285904 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d2628-ed11-4586-b8a2-47f6b62e88a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:45.287628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d2628-ed11-4586-b8a2-47f6b62e88a8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:45.289947 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 102/212] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:15:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:45.295274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d2628-ed11-4586-b8a2-47f6b62e88a8 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:45.299893 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 111/213] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:15:45.322545 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfa266-4f31-4e4f-9387-ced76dd56d0e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:45.401782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:45.424765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:45.440641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d2628-ed11-4586-b8a2-47f6b62e88a8 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:45.459147 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfa266-4f31-4e4f-9387-ced76dd56d0e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:45.499819 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f1acf-ecfe-4a1c-ad2c-e9e91ba8eceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:45.500593 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f1acf-ecfe-4a1c-ad2c-e9e91ba8eceb None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:45.500898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f1acf-ecfe-4a1c-ad2c-e9e91ba8eceb None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:45.501185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f1acf-ecfe-4a1c-ad2c-e9e91ba8eceb None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:45.502136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f1acf-ecfe-4a1c-ad2c-e9e91ba8eceb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:45.503534 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f1acf-ecfe-4a1c-ad2c-e9e91ba8eceb None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:45.508827 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e288e-d4a5-4951-a0b2-b29b60811840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:45.510035 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e288e-d4a5-4951-a0b2-b29b60811840 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:45.510524 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e288e-d4a5-4951-a0b2-b29b60811840 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:45.511028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e288e-d4a5-4951-a0b2-b29b60811840 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:45.526621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e288e-d4a5-4951-a0b2-b29b60811840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:45.536806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-6e1e288e-d4a5-4951-a0b2-b29b60811840 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:45.536806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:45.536806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:45.536806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:45.536806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:45.536806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:45.536806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:45.536806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:45.536806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:45.536806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:45.536806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:45.536806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:45.536806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:45.536806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:45.536806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:45.536806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:45.538228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:45.538228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:45.538228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:45.538228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:45.538228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:45.538228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:45.538228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:45.538228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:45.538228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:45.538228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:45.538228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:45.538228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:15:45.538228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:45.538228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:45.538228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:45.538228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:45.539067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:45.539067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:45.539067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:45.539067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:45.539067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:45.539067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:45.539067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:45.539067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:45.539067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:45.539067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:45.539067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:45.539067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:15:45.539067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 112/214] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:15:45.549566 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e288e-d4a5-4951-a0b2-b29b60811840 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:45.549780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 103/215] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:15:45 2022] POST /identity/v3/users => generated 297 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:15:45.683261 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fb3ea-9e62-4530-9b47-4741834d55f5 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:45.683592 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fb3ea-9e62-4530-9b47-4741834d55f5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:45.683772 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fb3ea-9e62-4530-9b47-4741834d55f5 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:45.685565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 104/216] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:45.692831 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82333845-567e-4175-8b6f-9733bc62d38f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:45.693268 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82333845-567e-4175-8b6f-9733bc62d38f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:45.693690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82333845-567e-4175-8b6f-9733bc62d38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:45.716145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e288e-d4a5-4951-a0b2-b29b60811840 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:45.718083 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-82333845-567e-4175-8b6f-9733bc62d38f None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:45.719468 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-82333845-567e-4175-8b6f-9733bc62d38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:45.745901 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:45.758708 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cad2a7-d023-4f60-b698-9821d785d61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:45.759327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cad2a7-d023-4f60-b698-9821d785d61c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:45.759628 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cad2a7-d023-4f60-b698-9821d785d61c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:45.759917 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cad2a7-d023-4f60-b698-9821d785d61c None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:45.760827 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cad2a7-d023-4f60-b698-9821d785d61c None admin] RBAC: Adding query filter params (name=member) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:45.761145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cad2a7-d023-4f60-b698-9821d785d61c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:45.762650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cad2a7-d023-4f60-b698-9821d785d61c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:45.773739 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 113/217] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:15:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:45.781134 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cad2a7-d023-4f60-b698-9821d785d61c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:45.809952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 105/218] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:15:45.839791 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82333845-567e-4175-8b6f-9733bc62d38f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:45.926040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cad2a7-d023-4f60-b698-9821d785d61c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:45.980490 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:45.985718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575bfdd-9ee5-4851-a86d-e0160a71c7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:45.986628 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575bfdd-9ee5-4851-a86d-e0160a71c7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:45.986922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575bfdd-9ee5-4851-a86d-e0160a71c7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:45.987196 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575bfdd-9ee5-4851-a86d-e0160a71c7f3 None admin] PATH_INFO: `/v3/users/da39003e0f4142c2888a252856687175` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:46.008493 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575bfdd-9ee5-4851-a86d-e0160a71c7f3 None admin] RBAC: Authorizing `identity:get_user(user_id=da39003e0f4142c2888a252856687175)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:46.010397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575bfdd-9ee5-4851-a86d-e0160a71c7f3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:46.014471 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 114/219] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:45 2022] GET /identity/v3/users/da39003e0f4142c2888a252856687175 => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:46.018113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575bfdd-9ee5-4851-a86d-e0160a71c7f3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:46.035523 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82333845-567e-4175-8b6f-9733bc62d38f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:46.069839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:46.075902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb4fcb-13a5-404c-80d7-15f097d98b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:46.077476 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb4fcb-13a5-404c-80d7-15f097d98b19 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:46.077687 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb4fcb-13a5-404c-80d7-15f097d98b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:46.078033 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb4fcb-13a5-404c-80d7-15f097d98b19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:46.091856 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb4fcb-13a5-404c-80d7-15f097d98b19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:46.100909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-f5bb4fcb-13a5-404c-80d7-15f097d98b19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:46.100909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:46.100909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:46.100909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:46.100909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:46.100909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:46.100909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:46.100909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:46.100909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:46.100909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:46.100909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:46.100909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:46.100909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:46.100909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:46.100909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:46.100909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:46.102287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:46.102287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:46.102287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:46.102287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:46.102287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:46.102287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:46.102287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:46.102287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:46.102287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:46.102287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:46.102287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:46.102287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:15:46.102287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:46.102287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:46.102287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:46.102287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:46.103305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:46.103305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:46.103305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:46.103305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:46.103305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:46.103305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:46.103305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:46.103305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:46.103305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:46.103305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:46.103305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:46.103305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:15:46.103305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 106/220] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:15:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:15:46.118916 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb4fcb-13a5-404c-80d7-15f097d98b19 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:46.149511 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575bfdd-9ee5-4851-a86d-e0160a71c7f3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:46.196290 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59933771-ac17-41d3-af67-f1c741829507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:46.197032 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59933771-ac17-41d3-af67-f1c741829507 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:46.197369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59933771-ac17-41d3-af67-f1c741829507 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:46.197670 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59933771-ac17-41d3-af67-f1c741829507 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:46.294935 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59933771-ac17-41d3-af67-f1c741829507 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:46.299092 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59933771-ac17-41d3-af67-f1c741829507 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:46.441894 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:46.448183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb4fcb-13a5-404c-80d7-15f097d98b19 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:46.519109 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8cd2ac-ba5f-4ca8-a6a3-803f930e02e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:46.520257 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8cd2ac-ba5f-4ca8-a6a3-803f930e02e4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:46.520762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8cd2ac-ba5f-4ca8-a6a3-803f930e02e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:46.521259 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8cd2ac-ba5f-4ca8-a6a3-803f930e02e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:46.522502 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8cd2ac-ba5f-4ca8-a6a3-803f930e02e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:46.523057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8cd2ac-ba5f-4ca8-a6a3-803f930e02e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:46.525614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8cd2ac-ba5f-4ca8-a6a3-803f930e02e4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:46.537461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 115/221] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 06:15:46.539285 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:46.540112 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59933771-ac17-41d3-af67-f1c741829507 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:46.545258 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 107/222] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:46.555751 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8cd2ac-ba5f-4ca8-a6a3-803f930e02e4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:46.752329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8cd2ac-ba5f-4ca8-a6a3-803f930e02e4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:46.800882 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6799a-6538-4bac-a178-3f6cda620380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:46.801718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6799a-6538-4bac-a178-3f6cda620380 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:46.802003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6799a-6538-4bac-a178-3f6cda620380 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:46.802280 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6799a-6538-4bac-a178-3f6cda620380 None admin] PATH_INFO: `/v3/users/ea9d722d78de4f4fa7c1aceb35b551fc` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:46.808064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6799a-6538-4bac-a178-3f6cda620380 None admin] RBAC: Authorizing `identity:get_user(user_id=ea9d722d78de4f4fa7c1aceb35b551fc)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:46.809619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6799a-6538-4bac-a178-3f6cda620380 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:46.811981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 108/223] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:46 2022] GET /identity/v3/users/ea9d722d78de4f4fa7c1aceb35b551fc => generated 283 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:46.817806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6799a-6538-4bac-a178-3f6cda620380 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:46.837158 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59933771-ac17-41d3-af67-f1c741829507 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:46.923820 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025ac4c-55df-4124-9ab4-605d5c5d62e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:46.925794 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025ac4c-55df-4124-9ab4-605d5c5d62e2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:46.926594 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025ac4c-55df-4124-9ab4-605d5c5d62e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:46.927379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025ac4c-55df-4124-9ab4-605d5c5d62e2 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:46.929832 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025ac4c-55df-4124-9ab4-605d5c5d62e2 None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:46.941318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025ac4c-55df-4124-9ab4-605d5c5d62e2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:46.945244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 116/224] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:46 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:46.947940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ab7ad-b25d-46f7-86fa-c000e8dacaea None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:46.948747 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ab7ad-b25d-46f7-86fa-c000e8dacaea None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:46.949557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ab7ad-b25d-46f7-86fa-c000e8dacaea None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:46.950243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:46.953565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 117/225] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:46.957127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ab7ad-b25d-46f7-86fa-c000e8dacaea None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:46.975260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:46.996256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6799a-6538-4bac-a178-3f6cda620380 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:47.046312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb285886-ec16-4a86-8f95-ddc066ddc103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:47.047294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb285886-ec16-4a86-8f95-ddc066ddc103 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:47.047672 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb285886-ec16-4a86-8f95-ddc066ddc103 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:47.047834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb285886-ec16-4a86-8f95-ddc066ddc103 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:47.049113 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb285886-ec16-4a86-8f95-ddc066ddc103 None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:47.050686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb285886-ec16-4a86-8f95-ddc066ddc103 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:47.057403 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 109/226] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:46 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:47.057403 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554be41-0a18-424d-9c6b-60fbe1806d1f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:47.057403 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554be41-0a18-424d-9c6b-60fbe1806d1f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:47.057403 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554be41-0a18-424d-9c6b-60fbe1806d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:47.080296 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c554be41-0a18-424d-9c6b-60fbe1806d1f None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:47.082619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c554be41-0a18-424d-9c6b-60fbe1806d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:47.154204 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ab7ad-b25d-46f7-86fa-c000e8dacaea None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:47.184055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 110/227] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:47.186569 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554be41-0a18-424d-9c6b-60fbe1806d1f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:47.217911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe20511-5f0e-4a0f-8967-bb2056668ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:47.219054 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe20511-5f0e-4a0f-8967-bb2056668ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:47.220006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe20511-5f0e-4a0f-8967-bb2056668ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:47.220530 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe20511-5f0e-4a0f-8967-bb2056668ab1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:47.221717 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe20511-5f0e-4a0f-8967-bb2056668ab1 None admin] RBAC: Adding query filter params (role.id=ed6cf002b70f422190946ebcb114da60, scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78, user.id=da39003e0f4142c2888a252856687175) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:47.222328 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe20511-5f0e-4a0f-8967-bb2056668ab1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:47.224164 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe20511-5f0e-4a0f-8967-bb2056668ab1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:47.235567 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 118/228] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:15:46 2022] GET /identity/v3/role_assignments?user.id=da39003e0f4142c2888a252856687175&scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78&role.id=ed6cf002b70f422190946ebcb114da60 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:47.236209 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe20511-5f0e-4a0f-8967-bb2056668ab1 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:47.309242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:47.344302 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:47.374432 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554be41-0a18-424d-9c6b-60fbe1806d1f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:47.433674 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6b577-9605-4ad4-a749-de374b52d59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:47.434631 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6b577-9605-4ad4-a749-de374b52d59a None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:47.434919 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6b577-9605-4ad4-a749-de374b52d59a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:47.435201 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6b577-9605-4ad4-a749-de374b52d59a None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78/users/ea9d722d78de4f4fa7c1aceb35b551fc/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:47.442383 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6b577-9605-4ad4-a749-de374b52d59a None admin] RBAC: Authorizing `identity:create_grant(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78, user_id=ea9d722d78de4f4fa7c1aceb35b551fc, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:47.445823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe20511-5f0e-4a0f-8967-bb2056668ab1 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:47.448205 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6b577-9605-4ad4-a749-de374b52d59a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:47.474617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 111/229] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:15:47 2022] PUT /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78/users/ea9d722d78de4f4fa7c1aceb35b551fc/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:15:47.516341 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55ddfd-e09f-4782-9e94-f8e068f4bf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:47.517743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55ddfd-e09f-4782-9e94-f8e068f4bf86 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:47.518293 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55ddfd-e09f-4782-9e94-f8e068f4bf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:47.518782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55ddfd-e09f-4782-9e94-f8e068f4bf86 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:47.536501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55ddfd-e09f-4782-9e94-f8e068f4bf86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:47.546004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-aa55ddfd-e09f-4782-9e94-f8e068f4bf86 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:15:47.546004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:47.546004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:47.546004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:47.546004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:47.546004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:47.546004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:47.546004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:47.546004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:47.546004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:47.546004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:47.546004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:47.546004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:47.546004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:47.546004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:47.546004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:47.547683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:47.547683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:47.547683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:47.547683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:47.547683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:47.547683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:47.547683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:47.547683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:47.547683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:47.547683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:47.547683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:47.547683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:15:47.547683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:47.547683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:47.547683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:47.547683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:47.548525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:47.548525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:47.548525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:47.548525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:47.548525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:47.548525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:47.548525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:47.548525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:47.548525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:47.548525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:47.548525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:15:47.548525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:15:47.551652 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 119/230] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:15:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:47.558370 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6b577-9605-4ad4-a749-de374b52d59a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:47.741528 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6b577-9605-4ad4-a749-de374b52d59a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:47.775892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:47.789020 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334e67e-8f42-43a0-9e69-ebcba0033d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:47.791191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334e67e-8f42-43a0-9e69-ebcba0033d6c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:47.791622 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334e67e-8f42-43a0-9e69-ebcba0033d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:47.792283 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334e67e-8f42-43a0-9e69-ebcba0033d6c None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:47.793536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334e67e-8f42-43a0-9e69-ebcba0033d6c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:47.793928 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334e67e-8f42-43a0-9e69-ebcba0033d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:47.796214 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334e67e-8f42-43a0-9e69-ebcba0033d6c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:47.805453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 112/231] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:15:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:47.811495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55ddfd-e09f-4782-9e94-f8e068f4bf86 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:47.940373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55ddfd-e09f-4782-9e94-f8e068f4bf86 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:47.981487 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:47.987360 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15d684-dfda-4208-a6e1-b3b38a23c4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:47.988371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15d684-dfda-4208-a6e1-b3b38a23c4da None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:47.988658 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15d684-dfda-4208-a6e1-b3b38a23c4da None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:47.988940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15d684-dfda-4208-a6e1-b3b38a23c4da None admin] PATH_INFO: `/v3/users/ea9d722d78de4f4fa7c1aceb35b551fc` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:47.990269 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15d684-dfda-4208-a6e1-b3b38a23c4da None admin] RBAC: Authorizing `identity:get_user(user_id=ea9d722d78de4f4fa7c1aceb35b551fc)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:47.991814 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15d684-dfda-4208-a6e1-b3b38a23c4da None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:47.993800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 120/232] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:47 2022] GET /identity/v3/users/ea9d722d78de4f4fa7c1aceb35b551fc => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:48.000090 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334e67e-8f42-43a0-9e69-ebcba0033d6c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:48.200058 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334e67e-8f42-43a0-9e69-ebcba0033d6c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:48.284504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdbe65-6417-488e-80c8-620e0d62a4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:48.286140 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdbe65-6417-488e-80c8-620e0d62a4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:48.286667 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdbe65-6417-488e-80c8-620e0d62a4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:48.287162 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdbe65-6417-488e-80c8-620e0d62a4f9 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:48.289415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bdbe65-6417-488e-80c8-620e0d62a4f9 None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:48.292171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bdbe65-6417-488e-80c8-620e0d62a4f9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:48.297308 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 113/233] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:47 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:48.306851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15d684-dfda-4208-a6e1-b3b38a23c4da None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:48.451384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15d684-dfda-4208-a6e1-b3b38a23c4da None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:48.497080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e751008-5fed-4211-8bd4-3dc83f4134b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:48.498036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e751008-5fed-4211-8bd4-3dc83f4134b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:48.498351 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e751008-5fed-4211-8bd4-3dc83f4134b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:48.499160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e751008-5fed-4211-8bd4-3dc83f4134b6 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78/users/ea9d722d78de4f4fa7c1aceb35b551fc/roles/6e5dce7d61b34704ad2cd32251fa4d34` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:48.502045 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e751008-5fed-4211-8bd4-3dc83f4134b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78, user_id=ea9d722d78de4f4fa7c1aceb35b551fc, role_id=6e5dce7d61b34704ad2cd32251fa4d34)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:48.503529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e751008-5fed-4211-8bd4-3dc83f4134b6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:48.528982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 121/234] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:15:48 2022] PUT /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78/users/ea9d722d78de4f4fa7c1aceb35b551fc/roles/6e5dce7d61b34704ad2cd32251fa4d34 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:15:48.794598 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cf866-161d-4af6-8ce8-c0d6fe974764 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:48.795114 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cf866-161d-4af6-8ce8-c0d6fe974764 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:48.797076 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cf866-161d-4af6-8ce8-c0d6fe974764 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:48.800295 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 114/235] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:48.806050 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244eeed0-3600-4286-ba44-11ae2b322034 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:48.806354 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244eeed0-3600-4286-ba44-11ae2b322034 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:48.806631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244eeed0-3600-4286-ba44-11ae2b322034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:48.832715 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-244eeed0-3600-4286-ba44-11ae2b322034 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:48.833971 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-244eeed0-3600-4286-ba44-11ae2b322034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:48.908379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:48.928421 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 122/236] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:48.955591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900fffd-517e-4475-93aa-fafd1a88f4cf None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:48.955931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900fffd-517e-4475-93aa-fafd1a88f4cf None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:48.956190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900fffd-517e-4475-93aa-fafd1a88f4cf None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:48.957620 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 115/237] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:15:48.963390 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589743e-fb52-4f45-a5e0-2dc7b40de317 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:48.963681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589743e-fb52-4f45-a5e0-2dc7b40de317 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:48.963967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589743e-fb52-4f45-a5e0-2dc7b40de317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:48.973099 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900fffd-517e-4475-93aa-fafd1a88f4cf None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:48.985918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-6589743e-fb52-4f45-a5e0-2dc7b40de317 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:48.987130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6589743e-fb52-4f45-a5e0-2dc7b40de317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:49.166700 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:49.222860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 123/238] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:49.246660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589743e-fb52-4f45-a5e0-2dc7b40de317 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:49.267669 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900fffd-517e-4475-93aa-fafd1a88f4cf None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:49.337125 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c0ed0-a791-4503-828a-2854a4286033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:49.337968 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c0ed0-a791-4503-828a-2854a4286033 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:49.338252 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c0ed0-a791-4503-828a-2854a4286033 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:49.338530 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c0ed0-a791-4503-828a-2854a4286033 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:49.340978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c0ed0-a791-4503-828a-2854a4286033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:49.343046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c0ed0-a791-4503-828a-2854a4286033 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:49.351977 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 116/239] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:15:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:49.357532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c0ed0-a791-4503-828a-2854a4286033 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:49.460436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589743e-fb52-4f45-a5e0-2dc7b40de317 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:49.516139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:49.528383 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df980f94-fe3b-40b6-bcec-fa6f895aefb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:49.529202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df980f94-fe3b-40b6-bcec-fa6f895aefb0 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:49.531379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df980f94-fe3b-40b6-bcec-fa6f895aefb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:49.531661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df980f94-fe3b-40b6-bcec-fa6f895aefb0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:49.553318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df980f94-fe3b-40b6-bcec-fa6f895aefb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:49.563876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-df980f94-fe3b-40b6-bcec-fa6f895aefb0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:15:49.563876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:49.563876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:49.563876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:49.563876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:49.563876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:49.563876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:49.563876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:49.563876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:49.563876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:49.563876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:49.563876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:49.563876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:49.563876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:49.563876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:49.563876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:49.565661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:49.565661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:49.565661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:49.565661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:49.565661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:49.565661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:49.565661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:49.565661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:49.565661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:49.565661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:49.565661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:49.565661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:15:49.565661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:49.565661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:49.565661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:49.565661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:49.567160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:49.567160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:49.567160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:49.567160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:49.567160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:49.567160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:49.567160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:49.567160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:49.567160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:49.567160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:49.567160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:15:49.567160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:15:49.573312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c0ed0-a791-4503-828a-2854a4286033 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:49.573650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 124/240] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:15:49.582365 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df980f94-fe3b-40b6-bcec-fa6f895aefb0 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:49.601784 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:49.626405 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a346d4d-29e3-4c6d-b6b1-0a932b955d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:49.627169 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a346d4d-29e3-4c6d-b6b1-0a932b955d6f None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:49.627453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a346d4d-29e3-4c6d-b6b1-0a932b955d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:49.627790 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a346d4d-29e3-4c6d-b6b1-0a932b955d6f None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:49.628541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a346d4d-29e3-4c6d-b6b1-0a932b955d6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:49.630099 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a346d4d-29e3-4c6d-b6b1-0a932b955d6f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:49.691136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 117/241] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:15:49 2022] POST /identity/v3/users => generated 293 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:15:49.723937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df980f94-fe3b-40b6-bcec-fa6f895aefb0 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:49.762959 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a8cab-eacd-4e0c-909a-e8933c8c284b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:49.763643 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a8cab-eacd-4e0c-909a-e8933c8c284b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:49.764466 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a8cab-eacd-4e0c-909a-e8933c8c284b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:49.764753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a8cab-eacd-4e0c-909a-e8933c8c284b None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:49.765595 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a8cab-eacd-4e0c-909a-e8933c8c284b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:49.765905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a8cab-eacd-4e0c-909a-e8933c8c284b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:49.767333 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a8cab-eacd-4e0c-909a-e8933c8c284b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:49.777233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 125/242] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:15:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:49.783971 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a346d4d-29e3-4c6d-b6b1-0a932b955d6f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:49.940493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a346d4d-29e3-4c6d-b6b1-0a932b955d6f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:49.983418 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157585f-c6c5-4e1c-be1c-f2e475448878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:49.984631 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157585f-c6c5-4e1c-be1c-f2e475448878 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:49.985095 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157585f-c6c5-4e1c-be1c-f2e475448878 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:49.985577 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157585f-c6c5-4e1c-be1c-f2e475448878 None admin] PATH_INFO: `/v3/users/57e89781c9e04600bdbf7f5a98ca78ff` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:50.003570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157585f-c6c5-4e1c-be1c-f2e475448878 None admin] RBAC: Authorizing `identity:get_user(user_id=57e89781c9e04600bdbf7f5a98ca78ff)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:50.005410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157585f-c6c5-4e1c-be1c-f2e475448878 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:50.007714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 118/243] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:49 2022] GET /identity/v3/users/57e89781c9e04600bdbf7f5a98ca78ff => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:50.014476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a8cab-eacd-4e0c-909a-e8933c8c284b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:50.137574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:50.270349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a8cab-eacd-4e0c-909a-e8933c8c284b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:50.347285 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9442c-b93b-437c-afda-e5507f698ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:50.348319 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9442c-b93b-437c-afda-e5507f698ffb None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:50.348817 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9442c-b93b-437c-afda-e5507f698ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:50.349342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9442c-b93b-437c-afda-e5507f698ffb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:50.364823 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a7f35-8547-47e8-b792-e7878f3ce6a3 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:50.365414 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a7f35-8547-47e8-b792-e7878f3ce6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:50.365414 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a7f35-8547-47e8-b792-e7878f3ce6a3 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:50.366811 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 119/244] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:15:50.372347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f6d07-66e8-4fc0-baeb-8af2feaa3bed None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:50.372651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f6d07-66e8-4fc0-baeb-8af2feaa3bed None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:50.372905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f6d07-66e8-4fc0-baeb-8af2feaa3bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:50.400906 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-af7f6d07-66e8-4fc0-baeb-8af2feaa3bed None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:50.402479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-af7f6d07-66e8-4fc0-baeb-8af2feaa3bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:50.475304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:50.483966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9442c-b93b-437c-afda-e5507f698ffb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:50.485532 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9442c-b93b-437c-afda-e5507f698ffb None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:50.554326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 120/245] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:15:50.569597 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:50.606430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f6d07-66e8-4fc0-baeb-8af2feaa3bed None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:50.685149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 126/246] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 06:15:50.695611 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9442c-b93b-437c-afda-e5507f698ffb None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:50.821680 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f6d07-66e8-4fc0-baeb-8af2feaa3bed None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:50.884925 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c8a2d-fed6-454d-aa00-abd9598907b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:50.886784 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c8a2d-fed6-454d-aa00-abd9598907b2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:50.887565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c8a2d-fed6-454d-aa00-abd9598907b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:50.888321 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c8a2d-fed6-454d-aa00-abd9598907b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:50.894217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9442c-b93b-437c-afda-e5507f698ffb None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:50.909177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c8a2d-fed6-454d-aa00-abd9598907b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:50.925244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-fe1c8a2d-fed6-454d-aa00-abd9598907b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:50.925244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:50.925244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:50.925244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:50.925244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:50.925244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:50.925244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:50.925244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:50.925244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:50.925244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:50.925244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:50.925244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:50.925244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:50.925244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:50.925244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:50.925244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:50.927061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:50.927061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:50.927061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:50.927061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:50.927061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:50.927061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:50.927061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:50.927061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:50.927061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:50.927061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:50.927061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:50.927061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:15:50.927061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:50.927061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:50.927061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:50.927061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:50.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:50.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:50.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:50.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:50.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:50.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:50.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:50.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:50.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:50.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:50.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:15:50.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:15:50.933667 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 121/247] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:15:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:50.944357 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c8a2d-fed6-454d-aa00-abd9598907b2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:50.986552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e4ffd-2039-4fe2-935b-26279f9b3c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:50.988065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e4ffd-2039-4fe2-935b-26279f9b3c10 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:50.988571 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e4ffd-2039-4fe2-935b-26279f9b3c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:50.989078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e4ffd-2039-4fe2-935b-26279f9b3c10 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:50.994570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e4ffd-2039-4fe2-935b-26279f9b3c10 None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:50.997395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e4ffd-2039-4fe2-935b-26279f9b3c10 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:51.006183 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 127/248] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:50 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:51.017605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e4ffd-2039-4fe2-935b-26279f9b3c10 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:51.048543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:51.060721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:51.308179 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e4ffd-2039-4fe2-935b-26279f9b3c10 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:51.340845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c8a2d-fed6-454d-aa00-abd9598907b2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:51.425136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d787a-08c6-4475-b62a-ec2a5cbe0b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:51.426811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d787a-08c6-4475-b62a-ec2a5cbe0b51 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:51.427611 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d787a-08c6-4475-b62a-ec2a5cbe0b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:51.428444 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d787a-08c6-4475-b62a-ec2a5cbe0b51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:51.432321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d787a-08c6-4475-b62a-ec2a5cbe0b51 None admin] RBAC: Adding query filter params (role.id=f4272c5247654dfdbd6abccbd56374ff, scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78, user.id=57e89781c9e04600bdbf7f5a98ca78ff) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:51.434899 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d787a-08c6-4475-b62a-ec2a5cbe0b51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:51.439105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d787a-08c6-4475-b62a-ec2a5cbe0b51 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:51.463018 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 128/249] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:15:51 2022] GET /identity/v3/role_assignments?user.id=57e89781c9e04600bdbf7f5a98ca78ff&scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78&role.id=f4272c5247654dfdbd6abccbd56374ff => generated 259 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:51.464713 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbfb6b-5033-4923-8ee6-1439e20abb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:51.467222 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbfb6b-5033-4923-8ee6-1439e20abb27 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:51.469328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbfb6b-5033-4923-8ee6-1439e20abb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:51.470221 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbfb6b-5033-4923-8ee6-1439e20abb27 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:51.472199 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbfb6b-5033-4923-8ee6-1439e20abb27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:51.473139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbfb6b-5033-4923-8ee6-1439e20abb27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:51.476263 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbfb6b-5033-4923-8ee6-1439e20abb27 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:51.499639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 122/250] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 555 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:51.509454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d787a-08c6-4475-b62a-ec2a5cbe0b51 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:51.659790 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d01a9-1e17-4ac3-9044-c72758431e93 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:51.660282 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d01a9-1e17-4ac3-9044-c72758431e93 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:51.660751 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d01a9-1e17-4ac3-9044-c72758431e93 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:51.667968 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 123/251] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:15:51.673838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:51.674277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47938dae-0ca0-4206-85bb-8756fd90ecf9 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:51.674789 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47938dae-0ca0-4206-85bb-8756fd90ecf9 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:51.677307 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47938dae-0ca0-4206-85bb-8756fd90ecf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:51.724836 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-47938dae-0ca0-4206-85bb-8756fd90ecf9 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:51.726606 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-47938dae-0ca0-4206-85bb-8756fd90ecf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:51.729764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d787a-08c6-4475-b62a-ec2a5cbe0b51 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:51.815411 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee95778-f0ea-4a3c-b6c5-6d3c9a9bc42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:51.816882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee95778-f0ea-4a3c-b6c5-6d3c9a9bc42f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:51.817378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee95778-f0ea-4a3c-b6c5-6d3c9a9bc42f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:51.817864 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee95778-f0ea-4a3c-b6c5-6d3c9a9bc42f None admin] PATH_INFO: `/v3/users/ea9d722d78de4f4fa7c1aceb35b551fc` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:51.822790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee95778-f0ea-4a3c-b6c5-6d3c9a9bc42f None admin] RBAC: Authorizing `identity:get_user(user_id=ea9d722d78de4f4fa7c1aceb35b551fc)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:51.825621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee95778-f0ea-4a3c-b6c5-6d3c9a9bc42f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:51.829192 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 129/252] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:51 2022] GET /identity/v3/users/ea9d722d78de4f4fa7c1aceb35b551fc => generated 283 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:51.838933 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee95778-f0ea-4a3c-b6c5-6d3c9a9bc42f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:51.905739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 124/253] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:51.948658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47938dae-0ca0-4206-85bb-8756fd90ecf9 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:51.998099 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:52.080748 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee95778-f0ea-4a3c-b6c5-6d3c9a9bc42f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:52.168748 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf82261-80f1-4895-8d04-ec67d3b25a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:52.169875 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf82261-80f1-4895-8d04-ec67d3b25a4c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:52.170514 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf82261-80f1-4895-8d04-ec67d3b25a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:52.170863 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf82261-80f1-4895-8d04-ec67d3b25a4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:52.220781 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47938dae-0ca0-4206-85bb-8756fd90ecf9 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:52.225881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:52.356385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466cf98-9a88-4e84-9070-9c41b3009aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:52.359639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466cf98-9a88-4e84-9070-9c41b3009aef None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:52.360586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466cf98-9a88-4e84-9070-9c41b3009aef None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:52.361313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466cf98-9a88-4e84-9070-9c41b3009aef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:52.387610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf82261-80f1-4895-8d04-ec67d3b25a4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:52.391911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf82261-80f1-4895-8d04-ec67d3b25a4c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:52.406888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a466cf98-9a88-4e84-9070-9c41b3009aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:52.432575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-a466cf98-9a88-4e84-9070-9c41b3009aef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:52.432575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:52.432575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:52.432575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:52.432575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:52.432575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:52.432575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:52.432575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:52.432575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:52.432575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:52.432575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:52.432575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:52.432575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:52.432575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:52.432575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:52.432575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:52.434431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:52.434431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:52.434431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:52.434431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:52.434431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:52.434431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:52.434431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:52.434431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:52.434431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:52.434431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:52.434431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:52.434431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:15:52.434431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:52.434431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:52.434431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:52.434431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:52.436022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:52.436022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:52.436022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:52.436022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:52.436022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:52.436022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:52.436022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:52.436022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:52.436022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:52.436022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:52.436022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:52.436022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:15:52.441139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 125/254] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 493 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:15:52.450086 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466cf98-9a88-4e84-9070-9c41b3009aef None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:52.698392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:52.726639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466cf98-9a88-4e84-9070-9c41b3009aef None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:52.746768 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 130/255] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 909 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:15:52.748288 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef53be-c655-4311-9e4d-83e0b545b7b2 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:52.748741 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef53be-c655-4311-9e4d-83e0b545b7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:52.749607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef53be-c655-4311-9e4d-83e0b545b7b2 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:52.751041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 131/256] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:52.758454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef53be-c655-4311-9e4d-83e0b545b7b2 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:52.775503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3c5ac-5949-4573-8e8a-eefea32d98d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:52.776627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3c5ac-5949-4573-8e8a-eefea32d98d8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:52.777120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3c5ac-5949-4573-8e8a-eefea32d98d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:52.777668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3c5ac-5949-4573-8e8a-eefea32d98d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:52.779106 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3c5ac-5949-4573-8e8a-eefea32d98d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:52.780574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3c5ac-5949-4573-8e8a-eefea32d98d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:52.783244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3c5ac-5949-4573-8e8a-eefea32d98d8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:52.800283 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 126/257] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:15:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:52.802516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdacea-b50a-42ac-87e5-6d0d2aa1d328 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:52.802996 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdacea-b50a-42ac-87e5-6d0d2aa1d328 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:52.803480 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdacea-b50a-42ac-87e5-6d0d2aa1d328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:52.836846 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e7bdacea-b50a-42ac-87e5-6d0d2aa1d328 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:52.843220 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e7bdacea-b50a-42ac-87e5-6d0d2aa1d328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:52.850738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:52.973649 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef53be-c655-4311-9e4d-83e0b545b7b2 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:53.006130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 127/258] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:53.006607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdacea-b50a-42ac-87e5-6d0d2aa1d328 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:53.068461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e6bc0-78e0-4ac4-be7b-1062a30658c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:53.069440 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e6bc0-78e0-4ac4-be7b-1062a30658c1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:53.069725 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e6bc0-78e0-4ac4-be7b-1062a30658c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:53.070001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e6bc0-78e0-4ac4-be7b-1062a30658c1 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:53.073638 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e6bc0-78e0-4ac4-be7b-1062a30658c1 None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:53.075225 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e6bc0-78e0-4ac4-be7b-1062a30658c1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:53.078393 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 132/259] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:52 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:53.086406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e6bc0-78e0-4ac4-be7b-1062a30658c1 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:53.200117 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:53.290852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdacea-b50a-42ac-87e5-6d0d2aa1d328 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:53.370506 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21608fc7-8cae-4c34-9965-f21ae2ca5780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:53.372554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21608fc7-8cae-4c34-9965-f21ae2ca5780 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:53.373890 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21608fc7-8cae-4c34-9965-f21ae2ca5780 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:53.376176 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21608fc7-8cae-4c34-9965-f21ae2ca5780 None admin] PATH_INFO: `/v3/users/da39003e0f4142c2888a252856687175` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:53.383176 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21608fc7-8cae-4c34-9965-f21ae2ca5780 None admin] RBAC: Authorizing `identity:get_user(user_id=da39003e0f4142c2888a252856687175)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:53.386398 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21608fc7-8cae-4c34-9965-f21ae2ca5780 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:53.395093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 128/260] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:53 2022] GET /identity/v3/users/da39003e0f4142c2888a252856687175 => generated 297 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:53.401969 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21608fc7-8cae-4c34-9965-f21ae2ca5780 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:53.438488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e6bc0-78e0-4ac4-be7b-1062a30658c1 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:53.475566 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:53.544775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e55c4e-a855-412b-b329-f9e15a989f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:53.548341 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e55c4e-a855-412b-b329-f9e15a989f85 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:53.549264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e55c4e-a855-412b-b329-f9e15a989f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:53.550096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e55c4e-a855-412b-b329-f9e15a989f85 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:53.581573 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e55c4e-a855-412b-b329-f9e15a989f85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:53.593306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-e5e55c4e-a855-412b-b329-f9e15a989f85 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:15:53.593306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:53.593306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:53.593306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:53.593306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:53.593306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:53.593306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:53.593306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:53.593306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:53.593306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:53.593306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:53.593306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:53.593306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:53.593306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:53.593306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:53.593306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:53.595495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:53.595495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:53.595495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:53.595495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:53.595495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:53.595495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:53.595495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:53.595495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:53.595495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:53.595495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:53.595495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:53.595495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:15:53.595495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:53.595495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:53.595495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:53.595495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:53.597362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:53.597362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:53.597362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:53.597362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:53.597362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:53.597362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:53.597362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:53.597362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:53.597362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:53.597362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:53.597362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:15:53.597362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:15:53.597362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 133/261] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:15:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 509 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:53.597362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e55c4e-a855-412b-b329-f9e15a989f85 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:53.708190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21608fc7-8cae-4c34-9965-f21ae2ca5780 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:53.719250 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e55c4e-a855-412b-b329-f9e15a989f85 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:53.760542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055afdd-c7e0-4198-bab9-f18324869f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:53.762207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055afdd-c7e0-4198-bab9-f18324869f19 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:53.762652 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055afdd-c7e0-4198-bab9-f18324869f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:53.763083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055afdd-c7e0-4198-bab9-f18324869f19 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:53.765433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055afdd-c7e0-4198-bab9-f18324869f19 None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:53.767112 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055afdd-c7e0-4198-bab9-f18324869f19 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:53.769497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 134/262] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:53 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:53.770492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055afdd-c7e0-4198-bab9-f18324869f19 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:53.817631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:53.819320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed222c5-acc6-41c6-816a-32859b75cb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:53.820003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed222c5-acc6-41c6-816a-32859b75cb9f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:53.820479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed222c5-acc6-41c6-816a-32859b75cb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:53.820903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed222c5-acc6-41c6-816a-32859b75cb9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:53.821937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed222c5-acc6-41c6-816a-32859b75cb9f None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78, user.id=ea9d722d78de4f4fa7c1aceb35b551fc) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:53.822425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed222c5-acc6-41c6-816a-32859b75cb9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:53.824058 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed222c5-acc6-41c6-816a-32859b75cb9f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:53.840101 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 129/263] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:15:53 2022] GET /identity/v3/role_assignments?user.id=ea9d722d78de4f4fa7c1aceb35b551fc&scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 608 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:53.841227 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed222c5-acc6-41c6-816a-32859b75cb9f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:53.850848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:53.894342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055afdd-c7e0-4198-bab9-f18324869f19 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:53.938162 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87fd90-4bbd-435b-96c3-5c6d3a8ab631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:53.939224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87fd90-4bbd-435b-96c3-5c6d3a8ab631 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:53.939683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87fd90-4bbd-435b-96c3-5c6d3a8ab631 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:53.940147 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87fd90-4bbd-435b-96c3-5c6d3a8ab631 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:53.941115 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87fd90-4bbd-435b-96c3-5c6d3a8ab631 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:53.941631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87fd90-4bbd-435b-96c3-5c6d3a8ab631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:53.943307 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87fd90-4bbd-435b-96c3-5c6d3a8ab631 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:53.954441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 135/264] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:15:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:53.963215 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87fd90-4bbd-435b-96c3-5c6d3a8ab631 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:54.077525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed222c5-acc6-41c6-816a-32859b75cb9f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:54.148831 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c1e30-ca81-4a7c-98af-8907559ffd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:54.150813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c1e30-ca81-4a7c-98af-8907559ffd19 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:54.151506 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c1e30-ca81-4a7c-98af-8907559ffd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:54.152224 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c1e30-ca81-4a7c-98af-8907559ffd19 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78/users/da39003e0f4142c2888a252856687175/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:54.161059 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c1e30-ca81-4a7c-98af-8907559ffd19 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78, user_id=da39003e0f4142c2888a252856687175, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:54.164161 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c1e30-ca81-4a7c-98af-8907559ffd19 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:54.178694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87fd90-4bbd-435b-96c3-5c6d3a8ab631 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:54.182297 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:54.209705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 130/265] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:15:53 2022] PUT /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78/users/da39003e0f4142c2888a252856687175/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:15:54.275254 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af535f35-03ef-4e17-8d7d-a64af59b3062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:54.277103 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af535f35-03ef-4e17-8d7d-a64af59b3062 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:54.277676 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af535f35-03ef-4e17-8d7d-a64af59b3062 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:54.278238 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af535f35-03ef-4e17-8d7d-a64af59b3062 None admin] PATH_INFO: `/v3/users/ea9d722d78de4f4fa7c1aceb35b551fc` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:54.281107 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af535f35-03ef-4e17-8d7d-a64af59b3062 None admin] RBAC: Authorizing `identity:get_user(user_id=ea9d722d78de4f4fa7c1aceb35b551fc)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:54.288619 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af535f35-03ef-4e17-8d7d-a64af59b3062 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:54.292828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 136/266] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:53 2022] GET /identity/v3/users/ea9d722d78de4f4fa7c1aceb35b551fc => generated 283 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:54.302451 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c1e30-ca81-4a7c-98af-8907559ffd19 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:54.446611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:54.551057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c1e30-ca81-4a7c-98af-8907559ffd19 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:54.638407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a96af-ec61-40b9-997c-334337700494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:54.639655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a96af-ec61-40b9-997c-334337700494 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:54.639946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a96af-ec61-40b9-997c-334337700494 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:54.640226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a96af-ec61-40b9-997c-334337700494 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:54.729712 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a96af-ec61-40b9-997c-334337700494 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:54.731233 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a96af-ec61-40b9-997c-334337700494 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:54.817187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:54.859672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1597c-a875-4639-9cea-9c2152dd5869 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:54.860937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1597c-a875-4639-9cea-9c2152dd5869 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:54.865723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1597c-a875-4639-9cea-9c2152dd5869 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:54.869005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 137/267] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:54 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:15:54.874899 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 131/268] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:15:54.879108 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925a931-45b7-4e87-9320-2d954b8399fd None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:54.879988 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925a931-45b7-4e87-9320-2d954b8399fd None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:54.880747 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925a931-45b7-4e87-9320-2d954b8399fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:54.894542 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a96af-ec61-40b9-997c-334337700494 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:54.928124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-2925a931-45b7-4e87-9320-2d954b8399fd None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:54.929358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2925a931-45b7-4e87-9320-2d954b8399fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:55.041747 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 138/269] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:55.043841 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b85e19-34d3-4e11-be2c-99cfb3d715bb None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:55.044431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b85e19-34d3-4e11-be2c-99cfb3d715bb None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:55.045459 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b85e19-34d3-4e11-be2c-99cfb3d715bb None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:55.051853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 139/270] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:55.053752 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e76c9f-abaa-416b-a0a8-03605d473d62 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:55.054289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e76c9f-abaa-416b-a0a8-03605d473d62 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:55.054782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e76c9f-abaa-416b-a0a8-03605d473d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:55.074283 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-04e76c9f-abaa-416b-a0a8-03605d473d62 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:55.075968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-04e76c9f-abaa-416b-a0a8-03605d473d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:55.114504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a96af-ec61-40b9-997c-334337700494 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:55.165258 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f71d9c-6406-4904-9e56-5038a4a3cc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:55.166586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f71d9c-6406-4904-9e56-5038a4a3cc05 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:55.167102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f71d9c-6406-4904-9e56-5038a4a3cc05 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:55.167652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f71d9c-6406-4904-9e56-5038a4a3cc05 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:55.169183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f71d9c-6406-4904-9e56-5038a4a3cc05 None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:55.171519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f71d9c-6406-4904-9e56-5038a4a3cc05 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:55.173666 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 132/271] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:54 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:55.174699 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f71d9c-6406-4904-9e56-5038a4a3cc05 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:55.220784 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:55.223021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 140/272] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:55.229468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e76c9f-abaa-416b-a0a8-03605d473d62 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:55.283969 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:55.366962 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f71d9c-6406-4904-9e56-5038a4a3cc05 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:55.394368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e76c9f-abaa-416b-a0a8-03605d473d62 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:55.425804 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340934c-4134-42d0-aab0-7dd8bf2ecdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:55.426896 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340934c-4134-42d0-aab0-7dd8bf2ecdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:55.427311 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340934c-4134-42d0-aab0-7dd8bf2ecdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:55.427711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340934c-4134-42d0-aab0-7dd8bf2ecdb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:55.442155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5340934c-4134-42d0-aab0-7dd8bf2ecdb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:55.452326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-5340934c-4134-42d0-aab0-7dd8bf2ecdb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:55.452326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:55.452326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:55.452326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:55.452326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:55.452326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:55.452326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:55.452326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:55.452326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:55.452326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:55.452326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:55.452326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:55.452326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:55.452326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:55.452326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:55.452326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:55.454065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:55.454065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:55.454065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:55.454065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:55.454065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:55.454065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:55.454065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:55.454065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:55.454065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:55.454065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:55.454065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:55.454065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:15:55.454065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:55.454065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:55.454065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:55.454065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:55.456439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:55.456439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:55.456439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:55.456439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:55.456439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:55.456439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:55.456439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:55.456439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:55.456439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:55.456439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:55.456439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:55.456439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:15:55.456439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 133/273] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:55.456439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340934c-4134-42d0-aab0-7dd8bf2ecdb7 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:55.465548 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd946062-c718-43ef-8608-77e3094d31e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:55.466237 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd946062-c718-43ef-8608-77e3094d31e5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:55.466526 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd946062-c718-43ef-8608-77e3094d31e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:55.466858 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd946062-c718-43ef-8608-77e3094d31e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:55.467730 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd946062-c718-43ef-8608-77e3094d31e5 None admin] RBAC: Adding query filter params (role.id=6e5dce7d61b34704ad2cd32251fa4d34, scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78, user.id=ea9d722d78de4f4fa7c1aceb35b551fc) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:55.468239 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd946062-c718-43ef-8608-77e3094d31e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:55.469774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd946062-c718-43ef-8608-77e3094d31e5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:55.476760 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 141/274] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:15:55 2022] GET /identity/v3/role_assignments?user.id=ea9d722d78de4f4fa7c1aceb35b551fc&scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78&role.id=6e5dce7d61b34704ad2cd32251fa4d34 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:55.478120 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa8061-2756-45f2-b10d-a441072215bb None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:55.478410 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa8061-2756-45f2-b10d-a441072215bb None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:55.478696 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa8061-2756-45f2-b10d-a441072215bb None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:55.480037 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 142/275] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:55.480804 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa8061-2756-45f2-b10d-a441072215bb None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:55.637074 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:55.650522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340934c-4134-42d0-aab0-7dd8bf2ecdb7 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:55.664207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:55.678590 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa8061-2756-45f2-b10d-a441072215bb None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:55.727625 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c32db8-5af0-431d-a186-9794366e325b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:55.728378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c32db8-5af0-431d-a186-9794366e325b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:55.728666 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c32db8-5af0-431d-a186-9794366e325b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:55.729043 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c32db8-5af0-431d-a186-9794366e325b None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:55.729888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c32db8-5af0-431d-a186-9794366e325b None admin] RBAC: Adding query filter params (name=member) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:55.730056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314fad3-4382-40a1-842d-1fbddf030ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:55.730660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c32db8-5af0-431d-a186-9794366e325b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:55.730922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314fad3-4382-40a1-842d-1fbddf030ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:55.731159 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314fad3-4382-40a1-842d-1fbddf030ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:55.731402 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314fad3-4382-40a1-842d-1fbddf030ea1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:55.731642 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c32db8-5af0-431d-a186-9794366e325b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:55.733239 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4314fad3-4382-40a1-842d-1fbddf030ea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:55.735000 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4314fad3-4382-40a1-842d-1fbddf030ea1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:55.736854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 134/276] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:15:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:55.738568 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bd28b-3683-4182-ad94-c559f2ad4f30 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:55.738568 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bd28b-3683-4182-ad94-c559f2ad4f30 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:55.738568 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bd28b-3683-4182-ad94-c559f2ad4f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:55.747661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 143/277] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:15:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:55.748566 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c32db8-5af0-431d-a186-9794366e325b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:55.757698 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9f9bd28b-3683-4182-ad94-c559f2ad4f30 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:55.759559 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9f9bd28b-3683-4182-ad94-c559f2ad4f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:55.865900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 135/278] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:55.865900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bd28b-3683-4182-ad94-c559f2ad4f30 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:55.898094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c32db8-5af0-431d-a186-9794366e325b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:55.945370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f2567-53d6-463b-83ea-7669a772ef11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:55.946337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f2567-53d6-463b-83ea-7669a772ef11 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:55.946795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f2567-53d6-463b-83ea-7669a772ef11 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:55.947252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f2567-53d6-463b-83ea-7669a772ef11 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:55.948450 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f2567-53d6-463b-83ea-7669a772ef11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:55.950630 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f2567-53d6-463b-83ea-7669a772ef11 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:55.981404 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 144/279] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:15:55 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:15:55.983233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f2567-53d6-463b-83ea-7669a772ef11 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:56.002267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:56.028927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:56.053865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bd28b-3683-4182-ad94-c559f2ad4f30 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:56.129589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f2567-53d6-463b-83ea-7669a772ef11 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:56.133423 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08362b20-42d4-4be5-9560-ca971851bb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:56.134379 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08362b20-42d4-4be5-9560-ca971851bb0a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:56.134661 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08362b20-42d4-4be5-9560-ca971851bb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:56.134939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08362b20-42d4-4be5-9560-ca971851bb0a None admin] PATH_INFO: `/v3/users/c03bd6d6a2244626bb2dc084923e4263` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:56.171825 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08362b20-42d4-4be5-9560-ca971851bb0a None admin] RBAC: Authorizing `identity:get_user(user_id=c03bd6d6a2244626bb2dc084923e4263)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:56.176795 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08362b20-42d4-4be5-9560-ca971851bb0a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:56.177976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74246712-0385-45c5-bc90-96850bbf9f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:56.178826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74246712-0385-45c5-bc90-96850bbf9f9a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:56.179114 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74246712-0385-45c5-bc90-96850bbf9f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:56.179397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74246712-0385-45c5-bc90-96850bbf9f9a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:56.181773 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 136/280] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:55 2022] GET /identity/v3/users/c03bd6d6a2244626bb2dc084923e4263 => generated 293 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:56.189196 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08362b20-42d4-4be5-9560-ca971851bb0a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:56.203494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74246712-0385-45c5-bc90-96850bbf9f9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:56.212867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-74246712-0385-45c5-bc90-96850bbf9f9a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:15:56.212867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:56.212867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:56.212867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:56.212867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:56.212867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:56.212867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:56.212867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:56.212867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:56.212867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:56.212867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:56.212867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:56.212867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:56.212867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:56.212867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:56.212867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:56.222023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:56.222023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:56.222023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:56.222023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:56.222023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:56.222023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:56.222023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:56.222023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:56.222023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:56.222023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:56.222023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:56.222023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:15:56.222023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:56.222023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:56.222023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:56.222023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:56.223554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:56.223554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:56.223554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:56.223554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:56.223554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:56.223554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:56.223554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:56.223554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:56.223554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:56.223554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:56.223554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:15:56.223554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:15:56.223554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 145/281] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:56.227695 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74246712-0385-45c5-bc90-96850bbf9f9a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:56.336132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08362b20-42d4-4be5-9560-ca971851bb0a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:56.362615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:56.402497 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74c3c4-a427-4853-8790-ebb43c922e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:56.404053 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74c3c4-a427-4853-8790-ebb43c922e7b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:56.404685 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74c3c4-a427-4853-8790-ebb43c922e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:56.405523 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74c3c4-a427-4853-8790-ebb43c922e7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:56.407238 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74c3c4-a427-4853-8790-ebb43c922e7b None admin] RBAC: Adding query filter params (role.id=ed6cf002b70f422190946ebcb114da60, user.id=c03bd6d6a2244626bb2dc084923e4263, scope.system=all) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:56.407853 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74c3c4-a427-4853-8790-ebb43c922e7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:56.410654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74c3c4-a427-4853-8790-ebb43c922e7b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:56.416277 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74246712-0385-45c5-bc90-96850bbf9f9a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:56.420834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 137/282] 10.222.0.65 () {56 vars in 1308 bytes} [Thu May 19 06:15:56 2022] GET /identity/v3/role_assignments?user.id=c03bd6d6a2244626bb2dc084923e4263&scope.system=all&role.id=ed6cf002b70f422190946ebcb114da60 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:56.483504 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35640dcd-4074-40db-ade7-2b1e5e3d1f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:56.484442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35640dcd-4074-40db-ade7-2b1e5e3d1f7d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:56.484974 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35640dcd-4074-40db-ade7-2b1e5e3d1f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:56.485457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35640dcd-4074-40db-ade7-2b1e5e3d1f7d None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:56.486848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35640dcd-4074-40db-ade7-2b1e5e3d1f7d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:56.487323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35640dcd-4074-40db-ade7-2b1e5e3d1f7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:56.489797 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35640dcd-4074-40db-ade7-2b1e5e3d1f7d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:56.505546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 146/283] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:15:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:56.512485 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74c3c4-a427-4853-8790-ebb43c922e7b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:56.538936 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:56.683632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74c3c4-a427-4853-8790-ebb43c922e7b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:56.744316 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65983f90-5431-45fb-a137-0e778339448a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:56.745400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65983f90-5431-45fb-a137-0e778339448a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:56.745400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65983f90-5431-45fb-a137-0e778339448a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:56.745828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65983f90-5431-45fb-a137-0e778339448a None admin] PATH_INFO: `/v3/users/57e89781c9e04600bdbf7f5a98ca78ff` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:56.747029 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65983f90-5431-45fb-a137-0e778339448a None admin] RBAC: Authorizing `identity:get_user(user_id=57e89781c9e04600bdbf7f5a98ca78ff)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:56.749195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65983f90-5431-45fb-a137-0e778339448a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:56.751409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 138/284] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:56 2022] GET /identity/v3/users/57e89781c9e04600bdbf7f5a98ca78ff => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:56.759132 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35640dcd-4074-40db-ade7-2b1e5e3d1f7d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:56.994145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35640dcd-4074-40db-ade7-2b1e5e3d1f7d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:57.057833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:57.065112 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172c3ce-c170-42f5-aa61-381d2986fcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:57.066786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172c3ce-c170-42f5-aa61-381d2986fcf9 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:57.067309 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172c3ce-c170-42f5-aa61-381d2986fcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:57.067815 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172c3ce-c170-42f5-aa61-381d2986fcf9 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:57.070360 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172c3ce-c170-42f5-aa61-381d2986fcf9 None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:57.073930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172c3ce-c170-42f5-aa61-381d2986fcf9 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:57.077995 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 147/285] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:56 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:57.087724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65983f90-5431-45fb-a137-0e778339448a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:57.257768 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:57.315577 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65983f90-5431-45fb-a137-0e778339448a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:57.376602 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017b8fe-05cc-4630-ae36-900539315b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:57.377636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017b8fe-05cc-4630-ae36-900539315b53 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:57.377969 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017b8fe-05cc-4630-ae36-900539315b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:57.378254 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017b8fe-05cc-4630-ae36-900539315b53 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78/users/57e89781c9e04600bdbf7f5a98ca78ff/roles/f4272c5247654dfdbd6abccbd56374ff` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:57.383605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017b8fe-05cc-4630-ae36-900539315b53 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78, user_id=57e89781c9e04600bdbf7f5a98ca78ff, role_id=f4272c5247654dfdbd6abccbd56374ff)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:57.385109 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017b8fe-05cc-4630-ae36-900539315b53 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:57.408313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 139/286] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:15:57 2022] PUT /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78/users/57e89781c9e04600bdbf7f5a98ca78ff/roles/f4272c5247654dfdbd6abccbd56374ff => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:15:57.525110 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57921b-7b2a-4b02-9e4a-35f1cbc82950 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:57.525420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57921b-7b2a-4b02-9e4a-35f1cbc82950 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:57.525941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57921b-7b2a-4b02-9e4a-35f1cbc82950 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:57.527568 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 148/287] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:15:57.534379 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf129ce-5f74-41a9-bb8a-739fd43b02db None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:57.534982 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf129ce-5f74-41a9-bb8a-739fd43b02db None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:57.535478 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf129ce-5f74-41a9-bb8a-739fd43b02db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:57.554393 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-bcf129ce-5f74-41a9-bb8a-739fd43b02db None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:57.555960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bcf129ce-5f74-41a9-bb8a-739fd43b02db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:57.689365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 140/288] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:15:57.735203 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57921b-7b2a-4b02-9e4a-35f1cbc82950 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:57.946675 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57921b-7b2a-4b02-9e4a-35f1cbc82950 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:58.013090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750aaf2-07c5-4bb7-a5af-6da7370c4873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:58.015504 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750aaf2-07c5-4bb7-a5af-6da7370c4873 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:58.016020 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750aaf2-07c5-4bb7-a5af-6da7370c4873 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:58.016517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750aaf2-07c5-4bb7-a5af-6da7370c4873 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:58.038362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750aaf2-07c5-4bb7-a5af-6da7370c4873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:15:58.055002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-0750aaf2-07c5-4bb7-a5af-6da7370c4873 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:58.055002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:15:58.055002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:15:58.055002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:15:58.055002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:15:58.055002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:15:58.055002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:15:58.055002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:15:58.055002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:15:58.055002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:15:58.055002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:15:58.055002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:15:58.055002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:15:58.055002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:15:58.055002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:15:58.055002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:15:58.055967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:15:58.055967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:15:58.055967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:15:58.055967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:15:58.055967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:15:58.055967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:15:58.055967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:15:58.055967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:15:58.055967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:15:58.055967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:15:58.055967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:15:58.055967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:15:58.055967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:15:58.055967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:15:58.055967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:15:58.055967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:15:58.056761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:15:58.056761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:15:58.056761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:15:58.056761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:15:58.056761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:15:58.056761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:15:58.056761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:15:58.056761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:15:58.056761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:15:58.056761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:15:58.056761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:15:58.056761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:15:58.057969 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 149/289] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:15:58.066928 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf129ce-5f74-41a9-bb8a-739fd43b02db None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:58.231400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:58.290256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf129ce-5f74-41a9-bb8a-739fd43b02db None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:58.375343 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e05f26-44b4-4581-bcc1-f1a0c14e12e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:58.376408 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e05f26-44b4-4581-bcc1-f1a0c14e12e5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:58.376939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e05f26-44b4-4581-bcc1-f1a0c14e12e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:58.377456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e05f26-44b4-4581-bcc1-f1a0c14e12e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:58.378430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e05f26-44b4-4581-bcc1-f1a0c14e12e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:58.379046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e05f26-44b4-4581-bcc1-f1a0c14e12e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:58.380682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e05f26-44b4-4581-bcc1-f1a0c14e12e5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:58.397216 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 141/290] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:15:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:58.407241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750aaf2-07c5-4bb7-a5af-6da7370c4873 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:58.543730 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:58.621867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750aaf2-07c5-4bb7-a5af-6da7370c4873 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:58.695560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e762a9-e6f5-4861-be9b-392c586faee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:58.697236 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e762a9-e6f5-4861-be9b-392c586faee0 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:58.697764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e762a9-e6f5-4861-be9b-392c586faee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:58.698269 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e762a9-e6f5-4861-be9b-392c586faee0 None admin] PATH_INFO: `/v3/users/da39003e0f4142c2888a252856687175` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:58.702912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e762a9-e6f5-4861-be9b-392c586faee0 None admin] RBAC: Authorizing `identity:get_user(user_id=da39003e0f4142c2888a252856687175)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:58.705518 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e762a9-e6f5-4861-be9b-392c586faee0 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:58.708655 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 150/291] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:15:58 2022] GET /identity/v3/users/da39003e0f4142c2888a252856687175 => generated 297 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:58.717831 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e05f26-44b4-4581-bcc1-f1a0c14e12e5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:58.951378 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e05f26-44b4-4581-bcc1-f1a0c14e12e5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:59.024927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:59.037752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7650c-7eae-41b6-910d-7bcc1059cf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:59.038840 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7650c-7eae-41b6-910d-7bcc1059cf5b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:59.039342 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7650c-7eae-41b6-910d-7bcc1059cf5b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:59.039831 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7650c-7eae-41b6-910d-7bcc1059cf5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:59.215579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7650c-7eae-41b6-910d-7bcc1059cf5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:59.218747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7650c-7eae-41b6-910d-7bcc1059cf5b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:59.412529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 142/292] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:15:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 06:15:59.418074 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e762a9-e6f5-4861-be9b-392c586faee0 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:59.551721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e762a9-e6f5-4861-be9b-392c586faee0 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:59.571316 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd90c6-9379-4ae4-8f58-f0eeab4024f3 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:59.571981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd90c6-9379-4ae4-8f58-f0eeab4024f3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:59.572459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd90c6-9379-4ae4-8f58-f0eeab4024f3 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:59.574139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 143/293] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:15:59.580378 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feda6c9-6002-475d-ba8d-9b78385083e1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:59.580986 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feda6c9-6002-475d-ba8d-9b78385083e1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:59.581489 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feda6c9-6002-475d-ba8d-9b78385083e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:59.589681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:59.591631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7152a3-0466-4f70-b6e4-ac995085a69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:59.592555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7152a3-0466-4f70-b6e4-ac995085a69b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:59.592833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7152a3-0466-4f70-b6e4-ac995085a69b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:59.593524 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7152a3-0466-4f70-b6e4-ac995085a69b None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:59.595048 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7152a3-0466-4f70-b6e4-ac995085a69b None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:59.596716 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7152a3-0466-4f70-b6e4-ac995085a69b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:59.598951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 151/294] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:15:59 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:15:59.607720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7152a3-0466-4f70-b6e4-ac995085a69b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:59.628379 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2feda6c9-6002-475d-ba8d-9b78385083e1 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:59.629858 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2feda6c9-6002-475d-ba8d-9b78385083e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:59.692780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:59.745923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7152a3-0466-4f70-b6e4-ac995085a69b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:59.782790 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 144/295] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:15:59.784140 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c5d37-2834-49e5-93a2-d2357bf33542 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:59.784441 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c5d37-2834-49e5-93a2-d2357bf33542 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:59.784724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c5d37-2834-49e5-93a2-d2357bf33542 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:59.786449 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 145/296] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:15:59.790553 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271489f-6f53-4643-b9ef-1b00bd5e2cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:15:59.795675 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271489f-6f53-4643-b9ef-1b00bd5e2cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:59.796509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271489f-6f53-4643-b9ef-1b00bd5e2cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:59.797180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271489f-6f53-4643-b9ef-1b00bd5e2cb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:59.798299 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6271489f-6f53-4643-b9ef-1b00bd5e2cb3 None admin] RBAC: Adding query filter params (role.id=ed6cf002b70f422190946ebcb114da60, scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78, user.id=da39003e0f4142c2888a252856687175) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:15:59.798822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6271489f-6f53-4643-b9ef-1b00bd5e2cb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:15:59.800939 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6271489f-6f53-4643-b9ef-1b00bd5e2cb3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:15:59.801641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df002995-788c-48ff-ab12-095157855ccf None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:59.801994 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df002995-788c-48ff-ab12-095157855ccf None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:59.802223 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df002995-788c-48ff-ab12-095157855ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:59.809166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 152/297] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:15:59 2022] GET /identity/v3/role_assignments?user.id=da39003e0f4142c2888a252856687175&scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78&role.id=ed6cf002b70f422190946ebcb114da60 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:15:59.812057 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271489f-6f53-4643-b9ef-1b00bd5e2cb3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:15:59.839579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-df002995-788c-48ff-ab12-095157855ccf None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:15:59.842375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-df002995-788c-48ff-ab12-095157855ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:15:59.883020 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:15:59.941536 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271489f-6f53-4643-b9ef-1b00bd5e2cb3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:15:59.988980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 146/298] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:15:59.990724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a0e5e-9566-4ce5-a0a4-ea63fca113a1 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:15:59.991241 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a0e5e-9566-4ce5-a0a4-ea63fca113a1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:15:59.991717 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a0e5e-9566-4ce5-a0a4-ea63fca113a1 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:15:59.994528 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 147/299] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:15:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:00.002116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9182ea-6eb5-4ed1-a00d-95137cbdbe3e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:00.002909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9182ea-6eb5-4ed1-a00d-95137cbdbe3e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:00.003398 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9182ea-6eb5-4ed1-a00d-95137cbdbe3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:00.019686 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c188ef-fef4-482f-af86-772183059d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:00.021330 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c188ef-fef4-482f-af86-772183059d25 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:00.021608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c188ef-fef4-482f-af86-772183059d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:00.022121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c188ef-fef4-482f-af86-772183059d25 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:00.027060 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-6b9182ea-6eb5-4ed1-a00d-95137cbdbe3e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:00.028352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6b9182ea-6eb5-4ed1-a00d-95137cbdbe3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:00.042968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c188ef-fef4-482f-af86-772183059d25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:00.053139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-02c188ef-fef4-482f-af86-772183059d25 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:16:00.053139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:00.053139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:00.053139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:00.053139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:00.053139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:00.053139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:00.053139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:00.053139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:00.053139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:00.053139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:00.053139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:00.053139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:00.053139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:00.053139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:00.053139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:00.054247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:00.054247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:00.054247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:00.054247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:00.054247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:00.054247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:00.054247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:00.054247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:00.054247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:00.054247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:00.054247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:00.054247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:00.054247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:00.054247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:00.054247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:00.054247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:00.055233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:00.055233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:00.055233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:00.055233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:00.055233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:00.055233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:00.055233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:00.055233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:00.055233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:00.055233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:00.055233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:16:00.055233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:00.055233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 153/300] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:15:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:16:00.055233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c188ef-fef4-482f-af86-772183059d25 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:00.112852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 148/301] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:16:00.113989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9182ea-6eb5-4ed1-a00d-95137cbdbe3e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:00.142587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:00.200559 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c188ef-fef4-482f-af86-772183059d25 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:00.230381 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:00.252446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be6979-2d0e-42c4-aaed-5b011b796daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:00.253266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be6979-2d0e-42c4-aaed-5b011b796daa None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:00.253692 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be6979-2d0e-42c4-aaed-5b011b796daa None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:00.253954 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be6979-2d0e-42c4-aaed-5b011b796daa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:00.269133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9182ea-6eb5-4ed1-a00d-95137cbdbe3e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:00.269807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be6979-2d0e-42c4-aaed-5b011b796daa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:00.281264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-35be6979-2d0e-42c4-aaed-5b011b796daa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:16:00.281264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:00.281264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:00.281264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:00.281264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:00.281264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:00.281264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:00.281264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:00.281264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:00.281264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:00.281264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:00.281264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:00.281264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:00.281264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:00.281264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:00.281264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:00.282763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:00.282763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:00.282763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:00.282763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:00.282763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:00.282763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:00.282763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:00.282763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:00.282763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:00.282763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:00.282763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:00.282763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:00.282763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:00.282763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:00.282763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:00.282763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:00.284064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:00.284064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:00.284064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:00.284064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:00.284064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:00.284064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:00.284064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:00.284064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:00.284064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:00.284064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:00.284064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:16:00.284064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:00.284064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 154/302] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:00.284064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be6979-2d0e-42c4-aaed-5b011b796daa None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:00.338443 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc0f4e-564f-4216-ac9f-c475e6664ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:00.339774 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc0f4e-564f-4216-ac9f-c475e6664ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:00.340789 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc0f4e-564f-4216-ac9f-c475e6664ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:00.340789 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc0f4e-564f-4216-ac9f-c475e6664ed4 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:00.340789 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc0f4e-564f-4216-ac9f-c475e6664ed4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:00.340789 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc0f4e-564f-4216-ac9f-c475e6664ed4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:00.340789 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc0f4e-564f-4216-ac9f-c475e6664ed4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:00.374841 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 149/303] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:16:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:00.377202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc0f4e-564f-4216-ac9f-c475e6664ed4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:00.452346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be6979-2d0e-42c4-aaed-5b011b796daa None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:00.506001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b2f0b-4b77-4b18-98df-a0bc4889f94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:00.506885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b2f0b-4b77-4b18-98df-a0bc4889f94e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:00.507176 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b2f0b-4b77-4b18-98df-a0bc4889f94e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:00.507455 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b2f0b-4b77-4b18-98df-a0bc4889f94e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:00.509816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748b2f0b-4b77-4b18-98df-a0bc4889f94e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:00.511385 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748b2f0b-4b77-4b18-98df-a0bc4889f94e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:00.513484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 155/304] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:00.514245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b2f0b-4b77-4b18-98df-a0bc4889f94e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:00.551123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:00.583421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc0f4e-564f-4216-ac9f-c475e6664ed4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:00.585778 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:00.655988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95596b-e922-454f-a9ef-1a59697c8e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:00.657161 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95596b-e922-454f-a9ef-1a59697c8e58 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:00.657710 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95596b-e922-454f-a9ef-1a59697c8e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:00.658197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95596b-e922-454f-a9ef-1a59697c8e58 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:00.659376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b2f0b-4b77-4b18-98df-a0bc4889f94e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:00.659650 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95596b-e922-454f-a9ef-1a59697c8e58 None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:00.660184 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95596b-e922-454f-a9ef-1a59697c8e58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:00.662895 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95596b-e922-454f-a9ef-1a59697c8e58 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:00.683832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 150/305] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:16:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:00.683832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95596b-e922-454f-a9ef-1a59697c8e58 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:00.711488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa0e5b-8de5-4f80-abcf-617b500dcedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:00.712695 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa0e5b-8de5-4f80-abcf-617b500dcedd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:00.712973 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa0e5b-8de5-4f80-abcf-617b500dcedd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:00.713246 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa0e5b-8de5-4f80-abcf-617b500dcedd None admin] PATH_INFO: `/v3/users/b6147edcb80742fa85ff867801be1067` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:00.732111 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa0e5b-8de5-4f80-abcf-617b500dcedd None admin] RBAC: Authorizing `identity:get_user(user_id=b6147edcb80742fa85ff867801be1067)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:00.734090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa0e5b-8de5-4f80-abcf-617b500dcedd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:00.735877 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 156/306] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:16:00 2022] GET /identity/v3/users/b6147edcb80742fa85ff867801be1067 => generated 293 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:00.736677 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa0e5b-8de5-4f80-abcf-617b500dcedd None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:00.832120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95596b-e922-454f-a9ef-1a59697c8e58 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:00.883012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a39dd-93c3-4d01-973d-0b69f7219de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:00.883871 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a39dd-93c3-4d01-973d-0b69f7219de0 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:00.884341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a39dd-93c3-4d01-973d-0b69f7219de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:00.884754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a39dd-93c3-4d01-973d-0b69f7219de0 None admin] PATH_INFO: `/v3/groups` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:00.885914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a39dd-93c3-4d01-973d-0b69f7219de0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:00.887951 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a39dd-93c3-4d01-973d-0b69f7219de0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:00.893858 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa0e5b-8de5-4f80-abcf-617b500dcedd None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:00.905783 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:00.911585 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 151/307] 10.222.0.65 () {60 vars in 1134 bytes} [Thu May 19 06:16:00 2022] POST /identity/v3/groups => generated 229 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:16:00.913048 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a39dd-93c3-4d01-973d-0b69f7219de0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:00.934810 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:00.977695 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a8d01-3849-4dd7-9809-dc3084d603ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:00.980170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a8d01-3849-4dd7-9809-dc3084d603ad None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:00.981344 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a8d01-3849-4dd7-9809-dc3084d603ad None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:00.982054 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a8d01-3849-4dd7-9809-dc3084d603ad None admin] PATH_INFO: `/v3/users/c03bd6d6a2244626bb2dc084923e4263` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:00.985322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a8d01-3849-4dd7-9809-dc3084d603ad None admin] RBAC: Authorizing `identity:get_user(user_id=c03bd6d6a2244626bb2dc084923e4263)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:00.989755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a8d01-3849-4dd7-9809-dc3084d603ad None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:00.999447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 157/308] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:16:00 2022] GET /identity/v3/users/c03bd6d6a2244626bb2dc084923e4263 => generated 293 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:01.004456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a8d01-3849-4dd7-9809-dc3084d603ad None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:01.121304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a39dd-93c3-4d01-973d-0b69f7219de0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:01.181482 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27d184-3506-46b5-87c4-6d2be8043c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:01.182059 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27d184-3506-46b5-87c4-6d2be8043c10 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:01.182353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27d184-3506-46b5-87c4-6d2be8043c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:01.182641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27d184-3506-46b5-87c4-6d2be8043c10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:01.183637 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27d184-3506-46b5-87c4-6d2be8043c10 None admin] RBAC: Adding query filter params (role.id=f4272c5247654dfdbd6abccbd56374ff, user.id=b6147edcb80742fa85ff867801be1067, scope.system=all) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:01.183968 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27d184-3506-46b5-87c4-6d2be8043c10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:01.186168 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27d184-3506-46b5-87c4-6d2be8043c10 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:01.195485 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 152/309] 10.222.0.65 () {56 vars in 1308 bytes} [Thu May 19 06:16:00 2022] GET /identity/v3/role_assignments?user.id=b6147edcb80742fa85ff867801be1067&scope.system=all&role.id=f4272c5247654dfdbd6abccbd56374ff => generated 226 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:01.200276 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370dc55-bc12-47f1-ae56-15cfcd973b4f None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:01.202585 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370dc55-bc12-47f1-ae56-15cfcd973b4f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:01.202935 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a8d01-3849-4dd7-9809-dc3084d603ad None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:01.203143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370dc55-bc12-47f1-ae56-15cfcd973b4f None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:01.206511 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 153/310] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:16:01.215426 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935b792-f7e1-4a37-8bbe-3f9b40c05a2c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:01.217321 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935b792-f7e1-4a37-8bbe-3f9b40c05a2c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:01.218356 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935b792-f7e1-4a37-8bbe-3f9b40c05a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:01.250988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9935b792-f7e1-4a37-8bbe-3f9b40c05a2c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:01.253504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9935b792-f7e1-4a37-8bbe-3f9b40c05a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:01.268705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55b04c-13c1-4513-8e2f-09f9671295a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:01.270366 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55b04c-13c1-4513-8e2f-09f9671295a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:01.270912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55b04c-13c1-4513-8e2f-09f9671295a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:01.271325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55b04c-13c1-4513-8e2f-09f9671295a4 None admin] PATH_INFO: `/v3/system/users/c03bd6d6a2244626bb2dc084923e4263/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:01.274287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55b04c-13c1-4513-8e2f-09f9671295a4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c03bd6d6a2244626bb2dc084923e4263, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:01.276535 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55b04c-13c1-4513-8e2f-09f9671295a4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:01.298160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 158/311] 10.222.0.65 () {58 vars in 1334 bytes} [Thu May 19 06:16:01 2022] PUT /identity/v3/system/users/c03bd6d6a2244626bb2dc084923e4263/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:16:01.393111 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:01.401932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 154/312] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:16:01.447803 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55b04c-13c1-4513-8e2f-09f9671295a4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:01.539197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:01.633073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55b04c-13c1-4513-8e2f-09f9671295a4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:01.677835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe91bcb-e0a9-4f9a-a1f8-fa0b41617f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:01.678653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe91bcb-e0a9-4f9a-a1f8-fa0b41617f9e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:01.678929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe91bcb-e0a9-4f9a-a1f8-fa0b41617f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:01.679209 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe91bcb-e0a9-4f9a-a1f8-fa0b41617f9e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:01.692009 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe91bcb-e0a9-4f9a-a1f8-fa0b41617f9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:01.700017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-4fe91bcb-e0a9-4f9a-a1f8-fa0b41617f9e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:16:01.700017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:01.700017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:01.700017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:01.700017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:01.700017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:01.700017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:01.700017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:01.700017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:01.700017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:01.700017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:01.700017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:01.700017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:01.700017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:01.700017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:01.700017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:01.702408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:01.702408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:01.702408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:01.702408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:01.702408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:01.702408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:01.702408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:01.702408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:01.702408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:01.702408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:01.702408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:01.702408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:01.702408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:01.702408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:01.702408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:01.702408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:01.704581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:01.704581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:01.704581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:01.704581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:01.704581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:01.704581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:01.704581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:01.704581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:01.704581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:01.704581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:01.704581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:16:01.704581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:01.704581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 159/313] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:01.715840 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935b792-f7e1-4a37-8bbe-3f9b40c05a2c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:01.892871 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935b792-f7e1-4a37-8bbe-3f9b40c05a2c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:01.958116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd468492-01cb-4db7-b8a7-ce353241b637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:01.959181 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd468492-01cb-4db7-b8a7-ce353241b637 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:01.959677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd468492-01cb-4db7-b8a7-ce353241b637 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:01.960169 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd468492-01cb-4db7-b8a7-ce353241b637 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:01.961633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd468492-01cb-4db7-b8a7-ce353241b637 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:01.962145 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd468492-01cb-4db7-b8a7-ce353241b637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:01.964747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd468492-01cb-4db7-b8a7-ce353241b637 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:01.980666 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 155/314] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:16:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:01.987364 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe91bcb-e0a9-4f9a-a1f8-fa0b41617f9e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:02.141578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:02.146566 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe91bcb-e0a9-4f9a-a1f8-fa0b41617f9e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:02.190542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5fea5f-a543-400a-b74d-aa9338da43aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:02.191992 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fea5f-a543-400a-b74d-aa9338da43aa None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:02.192436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fea5f-a543-400a-b74d-aa9338da43aa None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:02.192902 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fea5f-a543-400a-b74d-aa9338da43aa None admin] PATH_INFO: `/v3/users/57e89781c9e04600bdbf7f5a98ca78ff` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:02.195151 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5fea5f-a543-400a-b74d-aa9338da43aa None admin] RBAC: Authorizing `identity:get_user(user_id=57e89781c9e04600bdbf7f5a98ca78ff)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:02.198060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5fea5f-a543-400a-b74d-aa9338da43aa None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:02.204420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 160/315] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:16:01 2022] GET /identity/v3/users/57e89781c9e04600bdbf7f5a98ca78ff => generated 297 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:02.213203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd468492-01cb-4db7-b8a7-ce353241b637 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:02.290604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:02.335725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd468492-01cb-4db7-b8a7-ce353241b637 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:02.381880 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a1f80-46e1-4aeb-b138-13d7abe0e242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:02.382546 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a1f80-46e1-4aeb-b138-13d7abe0e242 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:02.382829 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a1f80-46e1-4aeb-b138-13d7abe0e242 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:02.383119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a1f80-46e1-4aeb-b138-13d7abe0e242 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:02.462455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a1f80-46e1-4aeb-b138-13d7abe0e242 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:02.464037 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a1f80-46e1-4aeb-b138-13d7abe0e242 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:02.564177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:02.634840 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 156/316] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:02.643952 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5fea5f-a543-400a-b74d-aa9338da43aa None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:02.835232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5fea5f-a543-400a-b74d-aa9338da43aa None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:02.902667 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b4add-6837-460d-b50a-61e755375966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:02.904936 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b4add-6837-460d-b50a-61e755375966 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:02.906153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b4add-6837-460d-b50a-61e755375966 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:02.906930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b4add-6837-460d-b50a-61e755375966 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:02.909748 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b4add-6837-460d-b50a-61e755375966 None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:02.913135 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b4add-6837-460d-b50a-61e755375966 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:02.917027 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 161/317] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:16:02 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:02.927488 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a1f80-46e1-4aeb-b138-13d7abe0e242 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:03.108522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a1f80-46e1-4aeb-b138-13d7abe0e242 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:03.149522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48961b33-88ec-4752-9915-99b2c0d2011b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:03.150195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48961b33-88ec-4752-9915-99b2c0d2011b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:03.150484 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48961b33-88ec-4752-9915-99b2c0d2011b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:03.150761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48961b33-88ec-4752-9915-99b2c0d2011b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:03.151605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48961b33-88ec-4752-9915-99b2c0d2011b None admin] RBAC: Adding query filter params (role.id=f4272c5247654dfdbd6abccbd56374ff, scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78, user.id=57e89781c9e04600bdbf7f5a98ca78ff) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:03.151922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48961b33-88ec-4752-9915-99b2c0d2011b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:03.153674 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48961b33-88ec-4752-9915-99b2c0d2011b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:03.161535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 157/318] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:16:02 2022] GET /identity/v3/role_assignments?user.id=57e89781c9e04600bdbf7f5a98ca78ff&scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78&role.id=f4272c5247654dfdbd6abccbd56374ff => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:03.908643 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f68b4-0531-40f5-b3a8-7cbc0dbb1834 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:03.909715 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f68b4-0531-40f5-b3a8-7cbc0dbb1834 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:03.909715 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f68b4-0531-40f5-b3a8-7cbc0dbb1834 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:03.911007 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 162/319] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:16:03.917880 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa8a91-0151-493f-88ee-87fdf117e5f8 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:03.918440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa8a91-0151-493f-88ee-87fdf117e5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:03.919022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa8a91-0151-493f-88ee-87fdf117e5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:03.950483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-faaa8a91-0151-493f-88ee-87fdf117e5f8 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:03.953239 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-faaa8a91-0151-493f-88ee-87fdf117e5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:03.993917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:04.089360 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 158/320] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:16:04.100601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9d333-0336-4fd0-8320-c596d1ed7ddb None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:04.101111 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9d333-0336-4fd0-8320-c596d1ed7ddb None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:04.101651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9d333-0336-4fd0-8320-c596d1ed7ddb None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:04.104835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 163/321] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:04.111529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e392d-eb60-4f53-8a98-e05593472400 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:04.111849 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e392d-eb60-4f53-8a98-e05593472400 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:04.112125 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e392d-eb60-4f53-8a98-e05593472400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:04.130359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-835e392d-eb60-4f53-8a98-e05593472400 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:04.131362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9d333-0336-4fd0-8320-c596d1ed7ddb None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:04.131585 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-835e392d-eb60-4f53-8a98-e05593472400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:04.187947 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:04.223367 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 159/322] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:16:04.225571 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158b102-a441-404f-919f-40c6c814b1ca None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:04.226045 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158b102-a441-404f-919f-40c6c814b1ca None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:04.226505 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158b102-a441-404f-919f-40c6c814b1ca None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:04.228522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 160/323] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:04.236369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b4d2c-c51a-4c51-9784-3d8c5d6fac16 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:04.237171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b4d2c-c51a-4c51-9784-3d8c5d6fac16 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:04.237731 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b4d2c-c51a-4c51-9784-3d8c5d6fac16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:04.267523 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-117b4d2c-c51a-4c51-9784-3d8c5d6fac16 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:04.269654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-117b4d2c-c51a-4c51-9784-3d8c5d6fac16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:04.288965 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9d333-0336-4fd0-8320-c596d1ed7ddb None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:04.338208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacca2b2-b2ec-4ab2-b1ff-15619c0c42f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:04.339083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacca2b2-b2ec-4ab2-b1ff-15619c0c42f5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:04.339436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacca2b2-b2ec-4ab2-b1ff-15619c0c42f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:04.339776 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacca2b2-b2ec-4ab2-b1ff-15619c0c42f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:04.352548 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacca2b2-b2ec-4ab2-b1ff-15619c0c42f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:04.361187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-eacca2b2-b2ec-4ab2-b1ff-15619c0c42f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:16:04.361187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:04.361187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:04.361187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:04.361187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:04.361187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:04.361187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:04.361187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:04.361187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:04.361187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:04.361187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:04.361187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:04.361187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:04.361187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:04.361187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:04.361187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:04.362986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:04.362986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:04.362986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:04.362986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:04.362986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:04.362986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:04.362986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:04.362986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:04.362986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:04.362986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:04.362986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:04.362986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:04.362986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:04.362986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:04.362986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:04.362986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:04.364552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:04.364552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:04.364552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:04.364552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:04.364552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:04.364552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:04.364552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:04.364552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:04.364552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:04.364552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:04.364552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:16:04.364552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:04.364552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 164/324] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:04.364552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacca2b2-b2ec-4ab2-b1ff-15619c0c42f5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:04.377435 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 161/325] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:16:04.377435 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b4d2c-c51a-4c51-9784-3d8c5d6fac16 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:04.410613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:04.475885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:04.484138 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacca2b2-b2ec-4ab2-b1ff-15619c0c42f5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:04.520592 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95c1ef-7346-4f05-82a0-bc992dff60e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:04.521423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95c1ef-7346-4f05-82a0-bc992dff60e9 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:04.521711 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95c1ef-7346-4f05-82a0-bc992dff60e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:04.521992 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95c1ef-7346-4f05-82a0-bc992dff60e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:04.533635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95c1ef-7346-4f05-82a0-bc992dff60e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:04.541378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-4e95c1ef-7346-4f05-82a0-bc992dff60e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:16:04.541378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:04.541378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:04.541378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:04.541378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:04.541378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:04.541378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:04.541378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:04.541378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:04.541378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:04.541378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:04.541378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:04.541378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:04.541378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:04.541378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:04.541378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:04.543200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:04.543200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:04.543200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:04.543200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:04.543200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:04.543200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:04.543200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:04.543200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:04.543200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:04.543200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:04.543200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:04.543200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:04.543200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:04.543200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:04.543200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:04.543200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:04.544851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:04.544851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:04.544851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:04.544851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:04.544851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:04.544851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:04.544851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:04.544851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:04.544851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:04.544851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:04.544851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:16:04.544851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:04.544851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 165/326] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:04.544851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95c1ef-7346-4f05-82a0-bc992dff60e9 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:04.565580 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b4d2c-c51a-4c51-9784-3d8c5d6fac16 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:04.611859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655e5ec-73f6-4cc0-8b4e-96a745760817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:04.612539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655e5ec-73f6-4cc0-8b4e-96a745760817 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:04.612827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655e5ec-73f6-4cc0-8b4e-96a745760817 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:04.613105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655e5ec-73f6-4cc0-8b4e-96a745760817 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:04.613909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655e5ec-73f6-4cc0-8b4e-96a745760817 None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:04.614222 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655e5ec-73f6-4cc0-8b4e-96a745760817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:04.615645 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655e5ec-73f6-4cc0-8b4e-96a745760817 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:04.625368 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 162/327] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:16:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:04.626243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655e5ec-73f6-4cc0-8b4e-96a745760817 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:04.674455 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95c1ef-7346-4f05-82a0-bc992dff60e9 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:04.725303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e712dd-d1c3-480c-8111-0b8c3e9c644d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:04.726385 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e712dd-d1c3-480c-8111-0b8c3e9c644d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:04.726776 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e712dd-d1c3-480c-8111-0b8c3e9c644d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:04.727236 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e712dd-d1c3-480c-8111-0b8c3e9c644d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:04.729252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e712dd-d1c3-480c-8111-0b8c3e9c644d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:04.731890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e712dd-d1c3-480c-8111-0b8c3e9c644d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:04.734701 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 166/328] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:04.735737 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e712dd-d1c3-480c-8111-0b8c3e9c644d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:04.747339 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:04.781506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:04.784790 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655e5ec-73f6-4cc0-8b4e-96a745760817 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:04.835454 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9949b-aaa5-424c-9017-0e9f9b7e67df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:04.836519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9949b-aaa5-424c-9017-0e9f9b7e67df None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:04.836934 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9949b-aaa5-424c-9017-0e9f9b7e67df None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:04.837419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9949b-aaa5-424c-9017-0e9f9b7e67df None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:04.838723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9949b-aaa5-424c-9017-0e9f9b7e67df None admin] RBAC: Adding query filter params (name=reader) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:04.839241 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9949b-aaa5-424c-9017-0e9f9b7e67df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:04.842187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9949b-aaa5-424c-9017-0e9f9b7e67df None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:04.856257 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 163/329] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:16:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:04.858113 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9949b-aaa5-424c-9017-0e9f9b7e67df None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:04.901857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e712dd-d1c3-480c-8111-0b8c3e9c644d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:04.944165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c26ed-a539-42e5-984f-57f03e867429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:04.945515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c26ed-a539-42e5-984f-57f03e867429 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:04.945886 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c26ed-a539-42e5-984f-57f03e867429 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:04.946250 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c26ed-a539-42e5-984f-57f03e867429 None admin] PATH_INFO: `/v3/users/c03bd6d6a2244626bb2dc084923e4263` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:04.947787 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c26ed-a539-42e5-984f-57f03e867429 None admin] RBAC: Authorizing `identity:get_user(user_id=c03bd6d6a2244626bb2dc084923e4263)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:04.949395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c26ed-a539-42e5-984f-57f03e867429 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:04.951977 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 167/330] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:16:04 2022] GET /identity/v3/users/c03bd6d6a2244626bb2dc084923e4263 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:04.953733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c26ed-a539-42e5-984f-57f03e867429 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:04.991048 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9949b-aaa5-424c-9017-0e9f9b7e67df None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:05.046038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffc500-2fb7-4db0-b400-017cb0bd6edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:05.046730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffc500-2fb7-4db0-b400-017cb0bd6edd None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:05.047022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffc500-2fb7-4db0-b400-017cb0bd6edd None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:05.047300 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffc500-2fb7-4db0-b400-017cb0bd6edd None admin] PATH_INFO: `/v3/groups` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:05.048014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffc500-2fb7-4db0-b400-017cb0bd6edd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:05.049519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffc500-2fb7-4db0-b400-017cb0bd6edd None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:05.062760 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:05.068752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 164/331] 10.222.0.65 () {60 vars in 1134 bytes} [Thu May 19 06:16:04 2022] POST /identity/v3/groups => generated 226 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:16:05.071488 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffc500-2fb7-4db0-b400-017cb0bd6edd None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:05.101781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:05.124519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c26ed-a539-42e5-984f-57f03e867429 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:05.194916 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85dc84-0a88-478b-a91a-e35196323879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:05.196504 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85dc84-0a88-478b-a91a-e35196323879 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:05.196989 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85dc84-0a88-478b-a91a-e35196323879 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:05.197516 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85dc84-0a88-478b-a91a-e35196323879 None admin] PATH_INFO: `/v3/users/b6147edcb80742fa85ff867801be1067` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:05.199905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85dc84-0a88-478b-a91a-e35196323879 None admin] RBAC: Authorizing `identity:get_user(user_id=b6147edcb80742fa85ff867801be1067)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:05.203444 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85dc84-0a88-478b-a91a-e35196323879 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:05.214257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 168/332] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:16:04 2022] GET /identity/v3/users/b6147edcb80742fa85ff867801be1067 => generated 293 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:05.215673 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85dc84-0a88-478b-a91a-e35196323879 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:05.237506 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffc500-2fb7-4db0-b400-017cb0bd6edd None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:05.288659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac1074-3a1a-4b74-8179-4d5661a5d6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:05.289667 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac1074-3a1a-4b74-8179-4d5661a5d6ce None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:05.290146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac1074-3a1a-4b74-8179-4d5661a5d6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:05.290649 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac1074-3a1a-4b74-8179-4d5661a5d6ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:05.291942 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac1074-3a1a-4b74-8179-4d5661a5d6ce None admin] RBAC: Adding query filter params (role.id=ed6cf002b70f422190946ebcb114da60, user.id=c03bd6d6a2244626bb2dc084923e4263, scope.system=all) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:05.293636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac1074-3a1a-4b74-8179-4d5661a5d6ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:05.297419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac1074-3a1a-4b74-8179-4d5661a5d6ce None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:05.314876 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 165/333] 10.222.0.65 () {56 vars in 1308 bytes} [Thu May 19 06:16:05 2022] GET /identity/v3/role_assignments?user.id=c03bd6d6a2244626bb2dc084923e4263&scope.system=all&role.id=ed6cf002b70f422190946ebcb114da60 => generated 510 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:05.408168 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85dc84-0a88-478b-a91a-e35196323879 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:05.484840 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2bdd87-05d0-4ad7-b170-3c22ae1c418a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:05.486487 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2bdd87-05d0-4ad7-b170-3c22ae1c418a None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:05.486944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2bdd87-05d0-4ad7-b170-3c22ae1c418a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:05.487459 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2bdd87-05d0-4ad7-b170-3c22ae1c418a None admin] PATH_INFO: `/v3/system/users/b6147edcb80742fa85ff867801be1067/roles/f4272c5247654dfdbd6abccbd56374ff` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:05.490486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2bdd87-05d0-4ad7-b170-3c22ae1c418a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b6147edcb80742fa85ff867801be1067, role_id=f4272c5247654dfdbd6abccbd56374ff)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:05.493762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2bdd87-05d0-4ad7-b170-3c22ae1c418a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:05.508146 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 169/334] 10.222.0.65 () {58 vars in 1334 bytes} [Thu May 19 06:16:05 2022] PUT /identity/v3/system/users/b6147edcb80742fa85ff867801be1067/roles/f4272c5247654dfdbd6abccbd56374ff => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:16:07.946236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29feea45-de95-46b0-9f20-328364e504b8 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:07.947289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29feea45-de95-46b0-9f20-328364e504b8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:07.947910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29feea45-de95-46b0-9f20-328364e504b8 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:07.949951 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 166/335] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:07.959480 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080dbf57-f3d2-40fb-8f7d-b4d1f55a094f None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:07.959993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080dbf57-f3d2-40fb-8f7d-b4d1f55a094f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:07.960066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080dbf57-f3d2-40fb-8f7d-b4d1f55a094f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:07.990672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-080dbf57-f3d2-40fb-8f7d-b4d1f55a094f None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:07.992011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-080dbf57-f3d2-40fb-8f7d-b4d1f55a094f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:08.018776 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:08.097155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 170/336] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:16:08.146112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29feea45-de95-46b0-9f20-328364e504b8 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:08.230151 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:08.290227 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29feea45-de95-46b0-9f20-328364e504b8 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:08.357788 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb13f2-f83c-48d1-b495-af69b18c14c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:08.359155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb13f2-f83c-48d1-b495-af69b18c14c2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:08.359650 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb13f2-f83c-48d1-b495-af69b18c14c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:08.360148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb13f2-f83c-48d1-b495-af69b18c14c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:08.390371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb13f2-f83c-48d1-b495-af69b18c14c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:08.406175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-37cb13f2-f83c-48d1-b495-af69b18c14c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:16:08.406175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:08.406175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:08.406175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:08.406175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:08.406175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:08.406175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:08.406175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:08.406175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:08.406175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:08.406175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:08.406175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:08.406175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:08.406175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:08.406175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:08.406175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:08.407173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:08.407173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:08.407173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:08.407173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:08.407173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:08.407173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:08.407173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:08.407173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:08.407173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:08.407173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:08.407173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:08.407173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:08.407173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:08.407173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:08.407173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:08.407173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:08.407959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:08.407959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:08.407959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:08.407959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:08.407959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:08.407959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:08.407959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:08.407959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:08.407959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:08.407959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:08.407959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:16:08.407959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:08.408594 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 167/337] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:08.415389 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080dbf57-f3d2-40fb-8f7d-b4d1f55a094f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:08.591581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080dbf57-f3d2-40fb-8f7d-b4d1f55a094f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:08.640046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b73053-c31f-448d-97f1-19aad0587abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:08.641105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b73053-c31f-448d-97f1-19aad0587abd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:08.641620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b73053-c31f-448d-97f1-19aad0587abd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:08.642133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b73053-c31f-448d-97f1-19aad0587abd None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:08.643110 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b73053-c31f-448d-97f1-19aad0587abd None admin] RBAC: Adding query filter params (name=member) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:08.643626 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b73053-c31f-448d-97f1-19aad0587abd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:08.645255 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b73053-c31f-448d-97f1-19aad0587abd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:08.658503 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 171/338] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:16:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:08.665143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb13f2-f83c-48d1-b495-af69b18c14c2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:08.953882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273f39e-4bee-451c-9e37-5a3d0f0ffc3c None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:08.954359 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273f39e-4bee-451c-9e37-5a3d0f0ffc3c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:08.954530 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273f39e-4bee-451c-9e37-5a3d0f0ffc3c None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:08.956145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 172/339] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:08.963245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab5587-c39f-4cfd-99a8-3ac224f9f890 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:08.963850 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb13f2-f83c-48d1-b495-af69b18c14c2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:08.964663 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab5587-c39f-4cfd-99a8-3ac224f9f890 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:08.965149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab5587-c39f-4cfd-99a8-3ac224f9f890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:09.000132 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-71ab5587-c39f-4cfd-99a8-3ac224f9f890 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:09.001761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-71ab5587-c39f-4cfd-99a8-3ac224f9f890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:09.027330 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:09.034204 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:09.106451 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ff407-4935-43b7-860e-9867d8739a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:09.107901 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ff407-4935-43b7-860e-9867d8739a8a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:09.108416 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ff407-4935-43b7-860e-9867d8739a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:09.108793 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ff407-4935-43b7-860e-9867d8739a8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:09.288160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 173/340] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:16:09.342143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab5587-c39f-4cfd-99a8-3ac224f9f890 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:09.405944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ff407-4935-43b7-860e-9867d8739a8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:09.408601 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ff407-4935-43b7-860e-9867d8739a8a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:09.664560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab5587-c39f-4cfd-99a8-3ac224f9f890 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:09.664993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 168/341] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1001 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:09.673887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ff407-4935-43b7-860e-9867d8739a8a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:09.710958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:09.744885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5df8e7-5229-4b1d-9acc-ed0e0d9b7973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:09.746173 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5df8e7-5229-4b1d-9acc-ed0e0d9b7973 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:09.746592 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5df8e7-5229-4b1d-9acc-ed0e0d9b7973 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:09.746998 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5df8e7-5229-4b1d-9acc-ed0e0d9b7973 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:09.774519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5df8e7-5229-4b1d-9acc-ed0e0d9b7973 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:09.798260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-9a5df8e7-5229-4b1d-9acc-ed0e0d9b7973 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:16:09.798260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:09.798260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:09.798260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:09.798260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:09.798260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:09.798260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:09.798260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:09.798260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:09.798260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:09.798260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:09.798260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:09.798260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:09.798260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:09.798260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:09.798260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:09.799218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:09.799218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:09.799218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:09.799218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:09.799218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:09.799218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:09.799218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:09.799218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:09.799218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:09.799218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:09.799218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:09.799218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:09.799218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:09.799218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:09.799218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:09.799218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:09.800544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:09.800544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:09.800544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:09.800544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:09.800544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:09.800544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:09.800544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:09.800544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:09.800544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:09.800544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:09.800544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:16:09.800544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:09.800544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 174/342] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:16:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 459 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:09.812630 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5df8e7-5229-4b1d-9acc-ed0e0d9b7973 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:09.818530 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:09.867039 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ff407-4935-43b7-860e-9867d8739a8a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:09.948804 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6faec5-7f5c-4ee2-ae58-61a309fd5691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:09.949635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6faec5-7f5c-4ee2-ae58-61a309fd5691 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:09.949923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6faec5-7f5c-4ee2-ae58-61a309fd5691 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:09.950200 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6faec5-7f5c-4ee2-ae58-61a309fd5691 None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:09.953273 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6faec5-7f5c-4ee2-ae58-61a309fd5691 None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:09.954812 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6faec5-7f5c-4ee2-ae58-61a309fd5691 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:09.958689 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 169/343] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:16:09 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:16:09.964847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6faec5-7f5c-4ee2-ae58-61a309fd5691 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:10.061183 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5df8e7-5229-4b1d-9acc-ed0e0d9b7973 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:10.108958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:10.124233 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6faec5-7f5c-4ee2-ae58-61a309fd5691 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:10.170211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922729cd-b48e-4586-9484-26d011cf8482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:10.171233 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922729cd-b48e-4586-9484-26d011cf8482 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:10.171515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922729cd-b48e-4586-9484-26d011cf8482 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:10.171797 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922729cd-b48e-4586-9484-26d011cf8482 None admin] PATH_INFO: `/v3/groups/c19765ac480d491f889cab5cfb00ab26` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:10.180378 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922729cd-b48e-4586-9484-26d011cf8482 None admin] RBAC: Authorizing `identity:get_group(group_id=c19765ac480d491f889cab5cfb00ab26)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:10.182112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922729cd-b48e-4586-9484-26d011cf8482 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:10.184259 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 170/344] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 06:16:09 2022] GET /identity/v3/groups/c19765ac480d491f889cab5cfb00ab26 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:10.196093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922729cd-b48e-4586-9484-26d011cf8482 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:10.234629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829d6f5-2925-4baa-83c6-77b5c372fb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:10.235720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829d6f5-2925-4baa-83c6-77b5c372fb29 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:10.236229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829d6f5-2925-4baa-83c6-77b5c372fb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:10.236724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829d6f5-2925-4baa-83c6-77b5c372fb29 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:10.238206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829d6f5-2925-4baa-83c6-77b5c372fb29 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:10.238748 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829d6f5-2925-4baa-83c6-77b5c372fb29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:10.249634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829d6f5-2925-4baa-83c6-77b5c372fb29 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:10.293377 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 175/345] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 481 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:16:10.305750 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515b66b-4993-4298-96d1-acc713c2324d None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:10.306267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515b66b-4993-4298-96d1-acc713c2324d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:10.306778 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515b66b-4993-4298-96d1-acc713c2324d None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:10.309748 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 176/346] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:10.328640 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892377ff-1863-4b99-b500-3ecf1102bbe6 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:10.329145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892377ff-1863-4b99-b500-3ecf1102bbe6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:10.329672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892377ff-1863-4b99-b500-3ecf1102bbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:10.378420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-892377ff-1863-4b99-b500-3ecf1102bbe6 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:10.379628 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-892377ff-1863-4b99-b500-3ecf1102bbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:10.386198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922729cd-b48e-4586-9484-26d011cf8482 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:10.448732 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d068e-7971-4b9c-a699-24b1b245f949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:10.449500 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d068e-7971-4b9c-a699-24b1b245f949 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:10.449795 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d068e-7971-4b9c-a699-24b1b245f949 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:10.450069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d068e-7971-4b9c-a699-24b1b245f949 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:10.451644 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d068e-7971-4b9c-a699-24b1b245f949 None admin] RBAC: Adding query filter params (group.id=c19765ac480d491f889cab5cfb00ab26, role.id=ed6cf002b70f422190946ebcb114da60, scope.project.id=fda8b64de40a4fd698a2458618d1a582) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:10.451962 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d068e-7971-4b9c-a699-24b1b245f949 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:10.453518 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d068e-7971-4b9c-a699-24b1b245f949 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:10.472592 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 171/347] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 06:16:10 2022] GET /identity/v3/role_assignments?group.id=c19765ac480d491f889cab5cfb00ab26&scope.project.id=fda8b64de40a4fd698a2458618d1a582&role.id=ed6cf002b70f422190946ebcb114da60 => generated 260 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:10.473599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d068e-7971-4b9c-a699-24b1b245f949 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:10.529178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:10.584626 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:10.609384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 177/348] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:10.671909 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892377ff-1863-4b99-b500-3ecf1102bbe6 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:10.676544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d068e-7971-4b9c-a699-24b1b245f949 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:10.722429 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5db261-7621-45b5-8132-9b47b517327d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:10.723116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5db261-7621-45b5-8132-9b47b517327d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:10.723407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5db261-7621-45b5-8132-9b47b517327d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:10.723694 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5db261-7621-45b5-8132-9b47b517327d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:10.825203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5db261-7621-45b5-8132-9b47b517327d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:10.827164 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5db261-7621-45b5-8132-9b47b517327d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:10.846385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:10.945875 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 172/349] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:10.947292 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2236af-9640-4cce-81e4-3f63bf60c0e0 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:10.947584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2236af-9640-4cce-81e4-3f63bf60c0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:10.947867 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2236af-9640-4cce-81e4-3f63bf60c0e0 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:10.949316 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 173/350] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:10.955631 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2236af-9640-4cce-81e4-3f63bf60c0e0 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:11.013853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892377ff-1863-4b99-b500-3ecf1102bbe6 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:11.090968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab1c17-8fbf-4c1f-961f-a2c2209d4139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:11.092484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab1c17-8fbf-4c1f-961f-a2c2209d4139 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:11.093542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab1c17-8fbf-4c1f-961f-a2c2209d4139 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:11.093542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab1c17-8fbf-4c1f-961f-a2c2209d4139 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:11.113345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab1c17-8fbf-4c1f-961f-a2c2209d4139 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:11.121612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-32ab1c17-8fbf-4c1f-961f-a2c2209d4139 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:16:11.121612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:11.121612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:11.121612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:11.121612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:11.121612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:11.121612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:11.121612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:11.121612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:11.121612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:11.121612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:11.121612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:11.121612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:11.121612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:11.121612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:11.121612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:11.122653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:11.122653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:11.122653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:11.122653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:11.122653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:11.122653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:11.122653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:11.122653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:11.122653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:11.122653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:11.122653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:11.122653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:11.122653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:11.122653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:11.122653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:11.122653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:11.124058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:11.124058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:11.124058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:11.124058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:11.124058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:11.124058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:11.124058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:11.124058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:11.124058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:11.124058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:11.124058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 06:16:11.124058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:11.124058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 178/351] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 452 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:11.124749 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08604fea-a408-4342-93bb-b0bfa0069df0 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:11.125148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08604fea-a408-4342-93bb-b0bfa0069df0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:11.125411 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08604fea-a408-4342-93bb-b0bfa0069df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:11.145157 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-08604fea-a408-4342-93bb-b0bfa0069df0 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:11.146491 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-08604fea-a408-4342-93bb-b0bfa0069df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:11.167069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2236af-9640-4cce-81e4-3f63bf60c0e0 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:11.200137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:11.246296 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1578021-42c1-4165-ac45-8990c97c8c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:11.247180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1578021-42c1-4165-ac45-8990c97c8c9c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:11.250156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1578021-42c1-4165-ac45-8990c97c8c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:11.250490 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1578021-42c1-4165-ac45-8990c97c8c9c None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:11.253975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1578021-42c1-4165-ac45-8990c97c8c9c None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:11.255499 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1578021-42c1-4165-ac45-8990c97c8c9c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:11.259729 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 174/352] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:16:10 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:16:11.260562 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1578021-42c1-4165-ac45-8990c97c8c9c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:11.281697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:11.311343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 179/353] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:11.312241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08604fea-a408-4342-93bb-b0bfa0069df0 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:11.472858 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1578021-42c1-4165-ac45-8990c97c8c9c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:11.482587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08604fea-a408-4342-93bb-b0bfa0069df0 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:11.511968 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837193a1-d1cb-4daa-8421-450cbb6c6aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:11.512659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837193a1-d1cb-4daa-8421-450cbb6c6aad None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:11.512952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837193a1-d1cb-4daa-8421-450cbb6c6aad None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:11.513254 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837193a1-d1cb-4daa-8421-450cbb6c6aad None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:11.514184 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837193a1-d1cb-4daa-8421-450cbb6c6aad None admin] RBAC: Adding query filter params (name=reader) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:11.514416 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837193a1-d1cb-4daa-8421-450cbb6c6aad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:11.515803 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837193a1-d1cb-4daa-8421-450cbb6c6aad None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:11.527632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 175/354] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:16:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:11.528605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837193a1-d1cb-4daa-8421-450cbb6c6aad None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:11.540339 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5bfc6-dfd6-4132-b37e-cacfac2cb3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:11.541906 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5bfc6-dfd6-4132-b37e-cacfac2cb3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:11.542596 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5bfc6-dfd6-4132-b37e-cacfac2cb3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:11.543354 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5bfc6-dfd6-4132-b37e-cacfac2cb3c3 None admin] PATH_INFO: `/v3/groups/c19765ac480d491f889cab5cfb00ab26` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:11.547083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c5bfc6-dfd6-4132-b37e-cacfac2cb3c3 None admin] RBAC: Authorizing `identity:get_group(group_id=c19765ac480d491f889cab5cfb00ab26)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:11.549503 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c5bfc6-dfd6-4132-b37e-cacfac2cb3c3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:11.551724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 180/355] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 06:16:11 2022] GET /identity/v3/groups/c19765ac480d491f889cab5cfb00ab26 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:11.554020 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6298bd-f6c4-49e7-8d81-93a7bad9a7d6 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:11.554763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6298bd-f6c4-49e7-8d81-93a7bad9a7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:11.555250 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6298bd-f6c4-49e7-8d81-93a7bad9a7d6 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:11.559111 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 181/356] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:11.563245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6298bd-f6c4-49e7-8d81-93a7bad9a7d6 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:11.589328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:11.651782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:11.654700 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837193a1-d1cb-4daa-8421-450cbb6c6aad None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:11.694634 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1a742-62b2-499e-a917-dafe2f7f6f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:11.695462 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1a742-62b2-499e-a917-dafe2f7f6f0e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:11.695749 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1a742-62b2-499e-a917-dafe2f7f6f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:11.696032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1a742-62b2-499e-a917-dafe2f7f6f0e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:11.710680 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1a742-62b2-499e-a917-dafe2f7f6f0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:11.719057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-a9f1a742-62b2-499e-a917-dafe2f7f6f0e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:16:11.719057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:11.719057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:11.719057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:11.719057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:11.719057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:11.719057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:11.719057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:11.719057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:11.719057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:11.719057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:11.719057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:11.719057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:11.719057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:11.719057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:11.719057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:11.729656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:11.729656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:11.729656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:11.729656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:11.729656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:11.729656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:11.729656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:11.729656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:11.729656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:11.729656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:11.729656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:11.729656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:11.729656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:11.729656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:11.729656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:11.729656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:11.732739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:11.732739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:11.732739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:11.732739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:11.732739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:11.732739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:11.732739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:11.732739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:11.732739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:11.732739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:11.732739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:16:11.732739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:11.732739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 176/357] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:11.732739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1a742-62b2-499e-a917-dafe2f7f6f0e None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:11.741788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6298bd-f6c4-49e7-8d81-93a7bad9a7d6 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:11.793738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd116ac-2193-44ed-9671-e170327c690e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:11.795958 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd116ac-2193-44ed-9671-e170327c690e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:11.796471 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd116ac-2193-44ed-9671-e170327c690e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:11.796902 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd116ac-2193-44ed-9671-e170327c690e None admin] PATH_INFO: `/v3/users/b6147edcb80742fa85ff867801be1067` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:11.798485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd116ac-2193-44ed-9671-e170327c690e None admin] RBAC: Authorizing `identity:get_user(user_id=b6147edcb80742fa85ff867801be1067)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:11.800150 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd116ac-2193-44ed-9671-e170327c690e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:11.802163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 182/358] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:16:11 2022] GET /identity/v3/users/b6147edcb80742fa85ff867801be1067 => generated 293 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:11.803768 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f67e65-7104-49d5-9fff-711792e00013 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:11.804218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f67e65-7104-49d5-9fff-711792e00013 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:11.804645 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f67e65-7104-49d5-9fff-711792e00013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:11.824207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-23f67e65-7104-49d5-9fff-711792e00013 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:11.826849 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-23f67e65-7104-49d5-9fff-711792e00013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:11.871278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1a742-62b2-499e-a917-dafe2f7f6f0e None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:11.875029 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:11.914770 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abf624-7882-4eb9-8a89-198ddaa16153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:11.915631 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abf624-7882-4eb9-8a89-198ddaa16153 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:11.916061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abf624-7882-4eb9-8a89-198ddaa16153 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:11.916440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abf624-7882-4eb9-8a89-198ddaa16153 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:11.917430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7abf624-7882-4eb9-8a89-198ddaa16153 None admin] RBAC: Adding query filter params (group.id=c19765ac480d491f889cab5cfb00ab26, role.id=6e5dce7d61b34704ad2cd32251fa4d34, scope.project.id=fda8b64de40a4fd698a2458618d1a582) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:11.917854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7abf624-7882-4eb9-8a89-198ddaa16153 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:11.919533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7abf624-7882-4eb9-8a89-198ddaa16153 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:11.919903 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 183/359] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:11.921218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f67e65-7104-49d5-9fff-711792e00013 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:11.926900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 177/360] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 06:16:11 2022] GET /identity/v3/role_assignments?group.id=c19765ac480d491f889cab5cfb00ab26&scope.project.id=fda8b64de40a4fd698a2458618d1a582&role.id=6e5dce7d61b34704ad2cd32251fa4d34 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:11.927885 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abf624-7882-4eb9-8a89-198ddaa16153 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:11.988076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:12.051329 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f67e65-7104-49d5-9fff-711792e00013 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:12.096853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dc6f1-c2b1-4f46-bcba-638a4689dd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:12.097664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc6f1-c2b1-4f46-bcba-638a4689dd76 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:12.097861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc6f1-c2b1-4f46-bcba-638a4689dd76 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:12.098139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc6f1-c2b1-4f46-bcba-638a4689dd76 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:12.098929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39dc6f1-c2b1-4f46-bcba-638a4689dd76 None admin] RBAC: Adding query filter params (name=member) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:12.099239 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39dc6f1-c2b1-4f46-bcba-638a4689dd76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:12.100706 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39dc6f1-c2b1-4f46-bcba-638a4689dd76 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:12.104899 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abf624-7882-4eb9-8a89-198ddaa16153 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:12.111008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 184/361] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:16:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:12.111901 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dc6f1-c2b1-4f46-bcba-638a4689dd76 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:12.151942 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9d558-121c-4dc4-ba14-6b7ce43e0e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:12.152993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9d558-121c-4dc4-ba14-6b7ce43e0e62 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:12.153486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9d558-121c-4dc4-ba14-6b7ce43e0e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:12.153950 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9d558-121c-4dc4-ba14-6b7ce43e0e62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:12.154911 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9d558-121c-4dc4-ba14-6b7ce43e0e62 None admin] RBAC: Adding query filter params (role.id=f4272c5247654dfdbd6abccbd56374ff, user.id=b6147edcb80742fa85ff867801be1067, scope.system=all) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:12.155585 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9d558-121c-4dc4-ba14-6b7ce43e0e62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:12.157230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9d558-121c-4dc4-ba14-6b7ce43e0e62 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:12.179537 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 178/362] 10.222.0.65 () {56 vars in 1308 bytes} [Thu May 19 06:16:11 2022] GET /identity/v3/role_assignments?user.id=b6147edcb80742fa85ff867801be1067&scope.system=all&role.id=f4272c5247654dfdbd6abccbd56374ff => generated 510 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:12.181217 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449df75-bc5c-450e-882f-40367a7c59bd None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:12.181659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449df75-bc5c-450e-882f-40367a7c59bd None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:12.182048 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449df75-bc5c-450e-882f-40367a7c59bd None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:12.183848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 179/363] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:12.184738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449df75-bc5c-450e-882f-40367a7c59bd None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:12.249895 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:12.326442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dc6f1-c2b1-4f46-bcba-638a4689dd76 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:12.355108 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:12.377778 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79e75c-158e-44ab-9cc8-604a655e41a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:12.378631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e75c-158e-44ab-9cc8-604a655e41a7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:12.378922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e75c-158e-44ab-9cc8-604a655e41a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:12.379271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e75c-158e-44ab-9cc8-604a655e41a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:12.401955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79e75c-158e-44ab-9cc8-604a655e41a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:12.420124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-5b79e75c-158e-44ab-9cc8-604a655e41a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:16:12.420124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:12.420124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:12.420124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:12.420124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:12.420124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:12.420124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:12.420124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:12.420124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:12.420124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:12.420124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:12.420124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:12.420124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:12.420124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:12.420124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:12.420124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:12.421678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:12.421678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:12.421678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:12.421678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:12.421678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:12.421678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:12.421678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:12.421678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:12.421678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:12.421678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:12.421678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:12.421678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:12.421678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:12.421678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:12.421678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:12.421678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:12.422660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:12.422660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:12.422660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:12.422660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:12.422660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:12.422660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:12.422660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:12.422660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:12.422660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:12.422660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:12.422660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:16:12.422660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:12.422660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 185/364] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:16:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:12.423557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93daed-3587-4244-b14f-8e483805cede None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:12.423829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93daed-3587-4244-b14f-8e483805cede None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:12.424010 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93daed-3587-4244-b14f-8e483805cede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:12.443671 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449df75-bc5c-450e-882f-40367a7c59bd None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:12.454163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ce93daed-3587-4244-b14f-8e483805cede None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:12.456430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ce93daed-3587-4244-b14f-8e483805cede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:12.528031 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41256d48-41af-41a0-81a8-f7ccf5a3f787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:12.530808 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41256d48-41af-41a0-81a8-f7ccf5a3f787 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:12.531955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41256d48-41af-41a0-81a8-f7ccf5a3f787 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:12.532768 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41256d48-41af-41a0-81a8-f7ccf5a3f787 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:12.612069 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 186/365] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:12.613700 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93daed-3587-4244-b14f-8e483805cede None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:12.722227 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41256d48-41af-41a0-81a8-f7ccf5a3f787 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:12.724790 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41256d48-41af-41a0-81a8-f7ccf5a3f787 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:12.786291 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:12.835244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93daed-3587-4244-b14f-8e483805cede None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:12.839734 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:12.936185 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a00edd-b3bf-42b3-a890-f95b3dd78cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:12.937243 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a00edd-b3bf-42b3-a890-f95b3dd78cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:12.937782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a00edd-b3bf-42b3-a890-f95b3dd78cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:12.938773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a00edd-b3bf-42b3-a890-f95b3dd78cb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:12.939756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a00edd-b3bf-42b3-a890-f95b3dd78cb5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:12.940306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a00edd-b3bf-42b3-a890-f95b3dd78cb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:12.947239 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a00edd-b3bf-42b3-a890-f95b3dd78cb5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:12.970202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 187/366] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:12.971746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a00edd-b3bf-42b3-a890-f95b3dd78cb5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:12.989330 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 180/367] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 805 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:12.991601 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41256d48-41af-41a0-81a8-f7ccf5a3f787 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:13.179880 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a00edd-b3bf-42b3-a890-f95b3dd78cb5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:13.221812 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41256d48-41af-41a0-81a8-f7ccf5a3f787 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:13.255374 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a898cc-8c03-4818-8a68-a250fdeefce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:13.256642 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a898cc-8c03-4818-8a68-a250fdeefce2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:13.257117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a898cc-8c03-4818-8a68-a250fdeefce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:13.257857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a898cc-8c03-4818-8a68-a250fdeefce2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:13.277000 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:13.281163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a898cc-8c03-4818-8a68-a250fdeefce2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:13.298993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-69a898cc-8c03-4818-8a68-a250fdeefce2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:16:13.298993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:13.298993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:13.298993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:13.298993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:13.298993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:13.298993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:13.298993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:13.298993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:13.298993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:13.298993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:13.298993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:13.298993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:13.298993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:13.298993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:13.298993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:13.300756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:13.300756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:13.300756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:13.300756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:13.300756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:13.300756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:13.300756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:13.300756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:13.300756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:13.300756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:13.300756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:13.300756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:13.300756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:13.300756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:13.300756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:13.300756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:13.303268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:13.303268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:13.303268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:13.303268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:13.303268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:13.303268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:13.303268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:13.303268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:13.303268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:13.303268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:13.303268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:16:13.303268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:13.303268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 188/368] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:16:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:13.303268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a898cc-8c03-4818-8a68-a250fdeefce2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:13.305221 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16a03f-cd35-4835-99fd-8a8e34de15b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:13.306725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16a03f-cd35-4835-99fd-8a8e34de15b2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:13.307456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16a03f-cd35-4835-99fd-8a8e34de15b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:13.308178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16a03f-cd35-4835-99fd-8a8e34de15b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:13.358596 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:13.443671 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16a03f-cd35-4835-99fd-8a8e34de15b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:13.446818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16a03f-cd35-4835-99fd-8a8e34de15b2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:13.504563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a898cc-8c03-4818-8a68-a250fdeefce2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:13.570510 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9c0ac-a4d9-4fc7-9400-a8cb9e740c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:13.571914 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9c0ac-a4d9-4fc7-9400-a8cb9e740c34 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:13.572395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9c0ac-a4d9-4fc7-9400-a8cb9e740c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:13.572902 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9c0ac-a4d9-4fc7-9400-a8cb9e740c34 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:13.575115 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9c0ac-a4d9-4fc7-9400-a8cb9e740c34 None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:13.578350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9c0ac-a4d9-4fc7-9400-a8cb9e740c34 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:13.581898 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 189/369] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:16:13 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:13.583267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9c0ac-a4d9-4fc7-9400-a8cb9e740c34 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:13.640765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 181/370] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:13.642787 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16a03f-cd35-4835-99fd-8a8e34de15b2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:13.692606 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:13.780256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:13.795118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9c0ac-a4d9-4fc7-9400-a8cb9e740c34 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:13.840416 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16a03f-cd35-4835-99fd-8a8e34de15b2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:13.862985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1ce0a-1cb7-4f73-8af7-db14942d2893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:13.864774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1ce0a-1cb7-4f73-8af7-db14942d2893 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:13.865434 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1ce0a-1cb7-4f73-8af7-db14942d2893 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:13.866062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1ce0a-1cb7-4f73-8af7-db14942d2893 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:13.867595 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1ce0a-1cb7-4f73-8af7-db14942d2893 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:13.868293 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1ce0a-1cb7-4f73-8af7-db14942d2893 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:13.871102 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1ce0a-1cb7-4f73-8af7-db14942d2893 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:13.887709 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 190/371] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:13.890197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1ce0a-1cb7-4f73-8af7-db14942d2893 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:13.911353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23185a6b-caa9-4cee-a953-8672423e0778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:13.913190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23185a6b-caa9-4cee-a953-8672423e0778 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:13.913526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23185a6b-caa9-4cee-a953-8672423e0778 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:13.914099 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23185a6b-caa9-4cee-a953-8672423e0778 None admin] PATH_INFO: `/v3/groups/c19765ac480d491f889cab5cfb00ab26` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:13.916508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23185a6b-caa9-4cee-a953-8672423e0778 None admin] RBAC: Authorizing `identity:get_group(group_id=c19765ac480d491f889cab5cfb00ab26)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:13.919094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23185a6b-caa9-4cee-a953-8672423e0778 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:13.922329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 182/372] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 06:16:13 2022] GET /identity/v3/groups/c19765ac480d491f889cab5cfb00ab26 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:13.929365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23185a6b-caa9-4cee-a953-8672423e0778 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:14.098047 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1ce0a-1cb7-4f73-8af7-db14942d2893 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:14.115030 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:14.129167 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23185a6b-caa9-4cee-a953-8672423e0778 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:14.169050 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18423bfb-710a-463c-bc67-d0d71418c765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:14.170876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18423bfb-710a-463c-bc67-d0d71418c765 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:14.171696 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18423bfb-710a-463c-bc67-d0d71418c765 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:14.172390 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18423bfb-710a-463c-bc67-d0d71418c765 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:14.174910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18423bfb-710a-463c-bc67-d0d71418c765 None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:14.178038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18423bfb-710a-463c-bc67-d0d71418c765 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:14.181918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 191/373] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:16:13 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:14.184442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18423bfb-710a-463c-bc67-d0d71418c765 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:14.193164 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb3d34-b8d4-4712-a3b7-d01eb112f5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:14.193927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb3d34-b8d4-4712-a3b7-d01eb112f5ff None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:14.194483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb3d34-b8d4-4712-a3b7-d01eb112f5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:14.195034 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb3d34-b8d4-4712-a3b7-d01eb112f5ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:14.235467 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:14.326745 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb3d34-b8d4-4712-a3b7-d01eb112f5ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:14.337252 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb3d34-b8d4-4712-a3b7-d01eb112f5ff None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:14.370817 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18423bfb-710a-463c-bc67-d0d71418c765 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:14.421355 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e9e71-acd0-452f-919b-c48cda830b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:14.422501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e9e71-acd0-452f-919b-c48cda830b20 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:14.422797 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e9e71-acd0-452f-919b-c48cda830b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:14.423087 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e9e71-acd0-452f-919b-c48cda830b20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:14.423956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e9e71-acd0-452f-919b-c48cda830b20 None admin] RBAC: Adding query filter params (group.id=c19765ac480d491f889cab5cfb00ab26, role.id=ed6cf002b70f422190946ebcb114da60, scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:14.424274 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e9e71-acd0-452f-919b-c48cda830b20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:14.425800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e9e71-acd0-452f-919b-c48cda830b20 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:14.432705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 192/374] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 06:16:14 2022] GET /identity/v3/role_assignments?group.id=c19765ac480d491f889cab5cfb00ab26&scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78&role.id=ed6cf002b70f422190946ebcb114da60 => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:14.433610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e9e71-acd0-452f-919b-c48cda830b20 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:14.517001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:14.528406 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 183/375] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:14.542388 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aeabca-d793-4b50-8ed3-25ab6fb7953c None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:14.543040 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aeabca-d793-4b50-8ed3-25ab6fb7953c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:14.543534 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aeabca-d793-4b50-8ed3-25ab6fb7953c None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:14.545869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 184/376] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:14 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:14.547046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aeabca-d793-4b50-8ed3-25ab6fb7953c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:14.587360 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:14.612515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e9e71-acd0-452f-919b-c48cda830b20 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:14.697854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cc5f0-e6f2-4dd5-894f-878edcc53510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:14.700335 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cc5f0-e6f2-4dd5-894f-878edcc53510 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:14.700335 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cc5f0-e6f2-4dd5-894f-878edcc53510 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:14.701226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cc5f0-e6f2-4dd5-894f-878edcc53510 None admin] PATH_INFO: `/v3/groups/c19765ac480d491f889cab5cfb00ab26` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:14.704656 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003cc5f0-e6f2-4dd5-894f-878edcc53510 None admin] RBAC: Authorizing `identity:get_group(group_id=c19765ac480d491f889cab5cfb00ab26)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:14.706393 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003cc5f0-e6f2-4dd5-894f-878edcc53510 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:14.708988 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 193/377] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 06:16:14 2022] GET /identity/v3/groups/c19765ac480d491f889cab5cfb00ab26 => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:14.711331 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6057704-5032-4b14-b987-48eea6baec98 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:14.711622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6057704-5032-4b14-b987-48eea6baec98 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:14.711910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6057704-5032-4b14-b987-48eea6baec98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:14.743363 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a6057704-5032-4b14-b987-48eea6baec98 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:14.744607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a6057704-5032-4b14-b987-48eea6baec98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:14.778465 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aeabca-d793-4b50-8ed3-25ab6fb7953c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:14.838596 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 194/378] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:14.839242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6057704-5032-4b14-b987-48eea6baec98 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:14.851655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4579248-d921-4288-8881-45f9d06bbfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:14.853420 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4579248-d921-4288-8881-45f9d06bbfcf None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:14.854025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4579248-d921-4288-8881-45f9d06bbfcf None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:14.854651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4579248-d921-4288-8881-45f9d06bbfcf None admin] PATH_INFO: `/v3/projects/0443648214584699a9ada0deda814964` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:14.859462 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4579248-d921-4288-8881-45f9d06bbfcf None admin] RBAC: Authorizing `identity:get_project(project_id=0443648214584699a9ada0deda814964)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:14.861658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4579248-d921-4288-8881-45f9d06bbfcf None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:14.865547 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 185/379] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:16:14 2022] GET /identity/v3/projects/0443648214584699a9ada0deda814964 => generated 344 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:14.874089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4579248-d921-4288-8881-45f9d06bbfcf None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:14.960641 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:14.963307 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:14.988579 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6057704-5032-4b14-b987-48eea6baec98 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:15.024974 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4579248-d921-4288-8881-45f9d06bbfcf None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:15.039177 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cc8db-4959-4808-897b-ee257ef35248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:15.040231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cc8db-4959-4808-897b-ee257ef35248 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:15.040688 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cc8db-4959-4808-897b-ee257ef35248 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:15.041125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cc8db-4959-4808-897b-ee257ef35248 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:15.042157 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1cc8db-4959-4808-897b-ee257ef35248 None admin] RBAC: Adding query filter params (group.id=c19765ac480d491f889cab5cfb00ab26, role.id=6e5dce7d61b34704ad2cd32251fa4d34, scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:15.042723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1cc8db-4959-4808-897b-ee257ef35248 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:15.045647 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1cc8db-4959-4808-897b-ee257ef35248 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:15.054472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 195/380] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 06:16:14 2022] GET /identity/v3/role_assignments?group.id=c19765ac480d491f889cab5cfb00ab26&scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78&role.id=6e5dce7d61b34704ad2cd32251fa4d34 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:15.055726 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cc8db-4959-4808-897b-ee257ef35248 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:15.077949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9a680-b3f3-4abe-8fef-d150ad5a7d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:15.078974 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9a680-b3f3-4abe-8fef-d150ad5a7d21 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:15.079357 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9a680-b3f3-4abe-8fef-d150ad5a7d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:15.079752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9a680-b3f3-4abe-8fef-d150ad5a7d21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:15.105615 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9a680-b3f3-4abe-8fef-d150ad5a7d21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:15.122104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-bde9a680-b3f3-4abe-8fef-d150ad5a7d21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:16:15.122104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:15.122104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:15.122104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:15.122104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:15.122104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:15.122104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:15.122104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:15.122104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:15.122104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:15.122104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:15.122104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:15.122104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:15.122104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:15.122104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:15.122104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:15.124022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:15.124022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:15.124022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:15.124022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:15.124022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:15.124022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:15.124022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:15.124022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:15.124022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:15.124022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:15.124022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:15.124022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:15.124022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:15.124022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:15.124022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:15.124022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:15.124949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:15.124949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:15.124949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:15.124949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:15.124949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:15.124949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:15.124949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:15.124949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:15.124949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:15.124949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:15.124949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:16:15.124949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:15.128485 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 186/381] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:15.139785 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9a680-b3f3-4abe-8fef-d150ad5a7d21 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:15.199407 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cc8db-4959-4808-897b-ee257ef35248 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:15.251680 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea521d0e-f07d-4dc9-a94a-c1d12fbb8128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:15.252762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea521d0e-f07d-4dc9-a94a-c1d12fbb8128 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:15.253153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea521d0e-f07d-4dc9-a94a-c1d12fbb8128 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:15.253561 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea521d0e-f07d-4dc9-a94a-c1d12fbb8128 None admin] PATH_INFO: `/v3/groups/7f2fe75d30e9411aaced865d57ea3acc` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:15.260774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea521d0e-f07d-4dc9-a94a-c1d12fbb8128 None admin] RBAC: Authorizing `identity:get_group(group_id=7f2fe75d30e9411aaced865d57ea3acc)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:15.262490 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea521d0e-f07d-4dc9-a94a-c1d12fbb8128 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:15.272456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 196/382] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 06:16:15 2022] GET /identity/v3/groups/7f2fe75d30e9411aaced865d57ea3acc => generated 229 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:15.273628 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea521d0e-f07d-4dc9-a94a-c1d12fbb8128 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:15.301050 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:15.358673 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9a680-b3f3-4abe-8fef-d150ad5a7d21 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:15.384538 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:15.410616 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861d9d3-6b5c-4617-acb6-dfa2a4eada46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:15.411693 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861d9d3-6b5c-4617-acb6-dfa2a4eada46 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:15.412197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861d9d3-6b5c-4617-acb6-dfa2a4eada46 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:15.412694 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861d9d3-6b5c-4617-acb6-dfa2a4eada46 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:15.414165 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861d9d3-6b5c-4617-acb6-dfa2a4eada46 None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:15.415114 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861d9d3-6b5c-4617-acb6-dfa2a4eada46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:15.417365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861d9d3-6b5c-4617-acb6-dfa2a4eada46 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:15.434288 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 187/383] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:16:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:15.437687 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912673cd-1d89-4ef2-b700-49c641cdeeda None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:15.438096 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912673cd-1d89-4ef2-b700-49c641cdeeda None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:15.438681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912673cd-1d89-4ef2-b700-49c641cdeeda None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:15.441569 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 188/384] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:15.443004 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912673cd-1d89-4ef2-b700-49c641cdeeda None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:15.462169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea521d0e-f07d-4dc9-a94a-c1d12fbb8128 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:15.509772 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e21ff-c58d-4a44-98c4-3ccd582458cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:15.510681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e21ff-c58d-4a44-98c4-3ccd582458cc None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:15.510974 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e21ff-c58d-4a44-98c4-3ccd582458cc None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:15.511379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e21ff-c58d-4a44-98c4-3ccd582458cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:15.512651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e21ff-c58d-4a44-98c4-3ccd582458cc None admin] RBAC: Adding query filter params (group.id=7f2fe75d30e9411aaced865d57ea3acc, role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=0443648214584699a9ada0deda814964) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:15.512977 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e21ff-c58d-4a44-98c4-3ccd582458cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:15.514588 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e21ff-c58d-4a44-98c4-3ccd582458cc None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:15.523634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 197/385] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 06:16:15 2022] GET /identity/v3/role_assignments?group.id=7f2fe75d30e9411aaced865d57ea3acc&scope.project.id=0443648214584699a9ada0deda814964&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 260 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:15.525029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdabab-7eb9-4a50-ba1f-12cb95f63d35 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:15.525372 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdabab-7eb9-4a50-ba1f-12cb95f63d35 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:15.525658 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdabab-7eb9-4a50-ba1f-12cb95f63d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:15.549195 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-89cdabab-7eb9-4a50-ba1f-12cb95f63d35 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:15.550592 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-89cdabab-7eb9-4a50-ba1f-12cb95f63d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:15.686856 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912673cd-1d89-4ef2-b700-49c641cdeeda None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:15.713002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 198/386] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:15.740513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdabab-7eb9-4a50-ba1f-12cb95f63d35 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:15.741644 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96034466-45bc-465f-8855-11478c7b6cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:15.742528 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96034466-45bc-465f-8855-11478c7b6cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:15.742861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96034466-45bc-465f-8855-11478c7b6cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:15.743144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96034466-45bc-465f-8855-11478c7b6cc0 None admin] PATH_INFO: `/v3/groups/c19765ac480d491f889cab5cfb00ab26` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:15.744916 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96034466-45bc-465f-8855-11478c7b6cc0 None admin] RBAC: Authorizing `identity:get_group(group_id=c19765ac480d491f889cab5cfb00ab26)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:15.746587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96034466-45bc-465f-8855-11478c7b6cc0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:15.752120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 189/387] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 06:16:15 2022] GET /identity/v3/groups/c19765ac480d491f889cab5cfb00ab26 => generated 226 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:15.754759 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96034466-45bc-465f-8855-11478c7b6cc0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:15.758846 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:15.829762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:15.908276 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96034466-45bc-465f-8855-11478c7b6cc0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:15.945436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdabab-7eb9-4a50-ba1f-12cb95f63d35 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:15.957667 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c5986-bfaa-44c1-b2fd-b14e977c806d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:15.959179 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c5986-bfaa-44c1-b2fd-b14e977c806d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:15.959685 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c5986-bfaa-44c1-b2fd-b14e977c806d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:15.960186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c5986-bfaa-44c1-b2fd-b14e977c806d None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:15.962408 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c5986-bfaa-44c1-b2fd-b14e977c806d None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:15.965167 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c5986-bfaa-44c1-b2fd-b14e977c806d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:15.970404 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 190/388] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:16:15 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:15.978987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c5986-bfaa-44c1-b2fd-b14e977c806d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:16.057308 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97abcd6-7b5c-457c-8e1a-c00a75acb665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:16.057308 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97abcd6-7b5c-457c-8e1a-c00a75acb665 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:16.057308 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97abcd6-7b5c-457c-8e1a-c00a75acb665 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:16.057308 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97abcd6-7b5c-457c-8e1a-c00a75acb665 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:16.115440 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97abcd6-7b5c-457c-8e1a-c00a75acb665 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:16.135180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-c97abcd6-7b5c-457c-8e1a-c00a75acb665 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:16:16.135180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:16.135180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:16.135180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:16.135180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:16.135180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:16.135180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:16.135180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:16.135180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:16.135180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:16.135180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:16.135180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:16.135180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:16.135180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:16.135180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:16.135180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:16.136148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:16.136148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:16.136148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:16.136148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:16.136148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:16.136148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:16.136148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:16.136148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:16.136148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:16.136148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:16.136148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:16.136148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:16.136148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:16.136148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:16.136148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:16.136148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:16.136941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:16.136941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:16.136941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:16.136941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:16.136941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:16.136941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:16.136941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:16.136941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:16.136941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:16.136941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:16.136941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:16:16.136941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:16.145325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 199/389] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:16:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:16.147118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97abcd6-7b5c-457c-8e1a-c00a75acb665 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:16.257180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c5986-bfaa-44c1-b2fd-b14e977c806d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:16.289965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:16.311638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c5a2f-5ddf-4bf7-9926-918676300747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:16.312795 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c5a2f-5ddf-4bf7-9926-918676300747 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:16.313154 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c5a2f-5ddf-4bf7-9926-918676300747 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:16.313669 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c5a2f-5ddf-4bf7-9926-918676300747 None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582/groups/c19765ac480d491f889cab5cfb00ab26/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:16.317476 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c5a2f-5ddf-4bf7-9926-918676300747 None admin] RBAC: Authorizing `identity:create_grant(project_id=fda8b64de40a4fd698a2458618d1a582, group_id=c19765ac480d491f889cab5cfb00ab26, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:16.321556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c5a2f-5ddf-4bf7-9926-918676300747 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:16.350924 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 191/390] 10.222.0.65 () {58 vars in 1442 bytes} [Thu May 19 06:16:15 2022] PUT /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582/groups/c19765ac480d491f889cab5cfb00ab26/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:16:16.438928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:16.501233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97abcd6-7b5c-457c-8e1a-c00a75acb665 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:16.561823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99789068-681c-49b0-866c-e331fbb44558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:16.562547 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99789068-681c-49b0-866c-e331fbb44558 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:16.562844 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99789068-681c-49b0-866c-e331fbb44558 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:16.563134 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99789068-681c-49b0-866c-e331fbb44558 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:16.563934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99789068-681c-49b0-866c-e331fbb44558 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:16.564252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99789068-681c-49b0-866c-e331fbb44558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:16.565763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99789068-681c-49b0-866c-e331fbb44558 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:16.593636 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 200/391] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:16.600477 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c5a2f-5ddf-4bf7-9926-918676300747 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:17.016556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c5a2f-5ddf-4bf7-9926-918676300747 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:17.075709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc46bbc-e451-481b-9bdc-fb66f7e7a0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:17.077028 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc46bbc-e451-481b-9bdc-fb66f7e7a0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:17.077566 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc46bbc-e451-481b-9bdc-fb66f7e7a0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:17.078057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc46bbc-e451-481b-9bdc-fb66f7e7a0b6 None admin] PATH_INFO: `/v3/groups/c19765ac480d491f889cab5cfb00ab26` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:17.082112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc46bbc-e451-481b-9bdc-fb66f7e7a0b6 None admin] RBAC: Authorizing `identity:get_group(group_id=c19765ac480d491f889cab5cfb00ab26)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:17.083912 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc46bbc-e451-481b-9bdc-fb66f7e7a0b6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:17.086293 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 192/392] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 06:16:16 2022] GET /identity/v3/groups/c19765ac480d491f889cab5cfb00ab26 => generated 226 bytes in 487 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:17.096441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99789068-681c-49b0-866c-e331fbb44558 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:17.325208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99789068-681c-49b0-866c-e331fbb44558 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:17.382127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:17.392203 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a9178-46dd-4fa6-be18-ee5fbace64e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:17.393728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a9178-46dd-4fa6-be18-ee5fbace64e2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:17.394697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a9178-46dd-4fa6-be18-ee5fbace64e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:17.395441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a9178-46dd-4fa6-be18-ee5fbace64e2 None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:17.398147 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a9178-46dd-4fa6-be18-ee5fbace64e2 None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:17.400878 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a9178-46dd-4fa6-be18-ee5fbace64e2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:17.404711 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 201/393] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:16:17 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:17.413356 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc46bbc-e451-481b-9bdc-fb66f7e7a0b6 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:17.509140 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:17.591678 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc46bbc-e451-481b-9bdc-fb66f7e7a0b6 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:17.651001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb4f94-3377-401b-aff0-254b2027a43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:17.652747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb4f94-3377-401b-aff0-254b2027a43a None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:17.653243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb4f94-3377-401b-aff0-254b2027a43a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:17.653786 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb4f94-3377-401b-aff0-254b2027a43a None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582/groups/c19765ac480d491f889cab5cfb00ab26/roles/6e5dce7d61b34704ad2cd32251fa4d34` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:17.659990 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb4f94-3377-401b-aff0-254b2027a43a None admin] RBAC: Authorizing `identity:create_grant(project_id=fda8b64de40a4fd698a2458618d1a582, group_id=c19765ac480d491f889cab5cfb00ab26, role_id=6e5dce7d61b34704ad2cd32251fa4d34)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:17.662979 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb4f94-3377-401b-aff0-254b2027a43a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:17.686959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 193/394] 10.222.0.65 () {58 vars in 1442 bytes} [Thu May 19 06:16:17 2022] PUT /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582/groups/c19765ac480d491f889cab5cfb00ab26/roles/6e5dce7d61b34704ad2cd32251fa4d34 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:16:17.868450 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117e7ed-ca38-4840-8711-065effd27e4c None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:17.869759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117e7ed-ca38-4840-8711-065effd27e4c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:17.870260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117e7ed-ca38-4840-8711-065effd27e4c None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:17.872813 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 202/395] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:17.878440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e850c-a929-46d4-88d7-bc8e28c73baf None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:17.879056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e850c-a929-46d4-88d7-bc8e28c73baf None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:17.879533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e850c-a929-46d4-88d7-bc8e28c73baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:17.899113 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f35e850c-a929-46d4-88d7-bc8e28c73baf None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:17.900712 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f35e850c-a929-46d4-88d7-bc8e28c73baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:17.988798 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 194/396] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:18.022502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117e7ed-ca38-4840-8711-065effd27e4c None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:18.148750 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117e7ed-ca38-4840-8711-065effd27e4c None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:18.221591 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0702a-bd77-4bc1-9db0-15f798ca584a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:18.223525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0702a-bd77-4bc1-9db0-15f798ca584a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:18.224360 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0702a-bd77-4bc1-9db0-15f798ca584a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:18.225139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0702a-bd77-4bc1-9db0-15f798ca584a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:18.247005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0702a-bd77-4bc1-9db0-15f798ca584a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:18.263694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-35d0702a-bd77-4bc1-9db0-15f798ca584a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:16:18.263694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:18.263694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:18.263694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:18.263694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:18.263694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:18.263694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:18.263694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:18.263694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:18.263694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:18.263694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:18.263694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:18.263694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:18.263694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:18.263694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:18.263694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:18.265621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:18.265621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:18.265621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:18.265621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:18.265621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:18.265621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:18.265621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:18.265621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:18.265621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:18.265621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:18.265621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:18.265621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:18.265621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:18.265621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:18.265621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:18.265621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:18.267235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:18.267235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:18.267235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:18.267235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:18.267235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:18.267235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:18.267235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:18.267235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:18.267235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:18.267235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:18.267235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:16:18.267235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:18.271847 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 203/397] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:18.282111 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e850c-a929-46d4-88d7-bc8e28c73baf None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:18.363100 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:18.541030 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e850c-a929-46d4-88d7-bc8e28c73baf None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:18.589468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34310119-0b3e-4641-9853-1326a401552c None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:18.590470 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34310119-0b3e-4641-9853-1326a401552c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:18.591225 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34310119-0b3e-4641-9853-1326a401552c None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:18.594654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 204/398] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:18.603495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0e36d-07a2-473a-a24a-8c8b78dcc6c7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:18.604443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0e36d-07a2-473a-a24a-8c8b78dcc6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:18.605251 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0e36d-07a2-473a-a24a-8c8b78dcc6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:18.635197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-92e0e36d-07a2-473a-a24a-8c8b78dcc6c7 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:18.637786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-92e0e36d-07a2-473a-a24a-8c8b78dcc6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:18.654588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59346a21-827a-4259-84f7-5b075a5c88c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:18.656013 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59346a21-827a-4259-84f7-5b075a5c88c3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:18.656524 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59346a21-827a-4259-84f7-5b075a5c88c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:18.657019 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59346a21-827a-4259-84f7-5b075a5c88c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:18.658521 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59346a21-827a-4259-84f7-5b075a5c88c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:18.659097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59346a21-827a-4259-84f7-5b075a5c88c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:18.661862 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59346a21-827a-4259-84f7-5b075a5c88c3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:18.690364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 195/399] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:16:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:18.705107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59346a21-827a-4259-84f7-5b075a5c88c3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:18.790366 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:18.857564 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 205/400] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:18.906278 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0e36d-07a2-473a-a24a-8c8b78dcc6c7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:19.053218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:19.062961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59346a21-827a-4259-84f7-5b075a5c88c3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:19.184287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba653c9-5604-46f9-be75-7785f5598158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:19.186410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba653c9-5604-46f9-be75-7785f5598158 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:19.187242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba653c9-5604-46f9-be75-7785f5598158 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:19.188036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba653c9-5604-46f9-be75-7785f5598158 None admin] PATH_INFO: `/v3/groups/c19765ac480d491f889cab5cfb00ab26` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:19.190664 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba653c9-5604-46f9-be75-7785f5598158 None admin] RBAC: Authorizing `identity:get_group(group_id=c19765ac480d491f889cab5cfb00ab26)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:19.192952 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0e36d-07a2-473a-a24a-8c8b78dcc6c7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:19.194607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba653c9-5604-46f9-be75-7785f5598158 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:19.198257 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 196/401] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 06:16:18 2022] GET /identity/v3/groups/c19765ac480d491f889cab5cfb00ab26 => generated 226 bytes in 494 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:16:19.207932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba653c9-5604-46f9-be75-7785f5598158 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:19.243012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f4f8b-3ba9-4a1e-9fab-cd8742bd55a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:19.243850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f4f8b-3ba9-4a1e-9fab-cd8742bd55a3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:19.244140 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f4f8b-3ba9-4a1e-9fab-cd8742bd55a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:19.244401 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f4f8b-3ba9-4a1e-9fab-cd8742bd55a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:19.262842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f4f8b-3ba9-4a1e-9fab-cd8742bd55a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:19.284170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-316f4f8b-3ba9-4a1e-9fab-cd8742bd55a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:16:19.284170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:19.284170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:19.284170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:19.284170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:19.284170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:19.284170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:19.284170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:19.284170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:19.284170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:19.284170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:19.284170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:19.284170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:19.284170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:19.284170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:19.284170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:19.285133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:19.285133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:19.285133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:19.285133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:19.285133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:19.285133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:19.285133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:19.285133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:19.285133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:19.285133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:19.285133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:19.285133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:19.285133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:19.285133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:19.285133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:19.285133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:19.285984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:19.285984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:19.285984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:19.285984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:19.285984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:19.285984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:19.285984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:19.285984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:19.285984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:19.285984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:19.285984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:16:19.285984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:19.288248 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 206/402] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:16:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:19.298023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f4f8b-3ba9-4a1e-9fab-cd8742bd55a3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:19.414743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:19.438879 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba653c9-5604-46f9-be75-7785f5598158 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:19.556094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783747ca-7878-48f9-8ebd-d08f4545e379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:19.558872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783747ca-7878-48f9-8ebd-d08f4545e379 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:19.561834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783747ca-7878-48f9-8ebd-d08f4545e379 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:19.562131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783747ca-7878-48f9-8ebd-d08f4545e379 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:19.566268 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783747ca-7878-48f9-8ebd-d08f4545e379 None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:19.568068 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783747ca-7878-48f9-8ebd-d08f4545e379 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:19.570536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 197/403] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:16:19 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:19.584156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783747ca-7878-48f9-8ebd-d08f4545e379 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:19.600912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f4f8b-3ba9-4a1e-9fab-cd8742bd55a3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:19.714529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:19.728826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7beffb9-d03c-4fee-9665-6cea0f8b840a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:19.731070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beffb9-d03c-4fee-9665-6cea0f8b840a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:19.732360 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beffb9-d03c-4fee-9665-6cea0f8b840a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:19.733397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beffb9-d03c-4fee-9665-6cea0f8b840a None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:19.735523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7beffb9-d03c-4fee-9665-6cea0f8b840a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:19.736330 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7beffb9-d03c-4fee-9665-6cea0f8b840a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:19.739186 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7beffb9-d03c-4fee-9665-6cea0f8b840a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:19.769877 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 207/404] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 466 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:19.770970 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230675d2-015a-45f7-8950-296f63329152 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:19.771265 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230675d2-015a-45f7-8950-296f63329152 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:19.771578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230675d2-015a-45f7-8950-296f63329152 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:19.773223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 208/405] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:19.773770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230675d2-015a-45f7-8950-296f63329152 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:19.829317 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783747ca-7878-48f9-8ebd-d08f4545e379 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:19.899733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230675d2-015a-45f7-8950-296f63329152 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:19.900405 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfec61-dc97-410b-bb84-c9fa44f80b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:19.902726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfec61-dc97-410b-bb84-c9fa44f80b8b None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:19.903654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfec61-dc97-410b-bb84-c9fa44f80b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:19.904640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfec61-dc97-410b-bb84-c9fa44f80b8b None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78/groups/c19765ac480d491f889cab5cfb00ab26/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:19.909243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bfec61-dc97-410b-bb84-c9fa44f80b8b None admin] RBAC: Authorizing `identity:create_grant(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78, group_id=c19765ac480d491f889cab5cfb00ab26, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:19.912480 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:19.915433 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bfec61-dc97-410b-bb84-c9fa44f80b8b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:19.946669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82794240-5dd0-4b9f-b52d-7f0e37abddc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:19.948205 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82794240-5dd0-4b9f-b52d-7f0e37abddc2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:19.948992 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82794240-5dd0-4b9f-b52d-7f0e37abddc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:19.949466 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82794240-5dd0-4b9f-b52d-7f0e37abddc2 None admin] PATH_INFO: `/v3/groups/c19765ac480d491f889cab5cfb00ab26` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:19.953469 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 198/406] 10.222.0.65 () {58 vars in 1442 bytes} [Thu May 19 06:16:19 2022] PUT /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78/groups/c19765ac480d491f889cab5cfb00ab26/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:16:19.956183 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82794240-5dd0-4b9f-b52d-7f0e37abddc2 None admin] RBAC: Authorizing `identity:get_group(group_id=c19765ac480d491f889cab5cfb00ab26)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:19.958700 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803b36a-1ef9-47dd-b790-741109e01fd1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:19.961960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803b36a-1ef9-47dd-b790-741109e01fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:19.962780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803b36a-1ef9-47dd-b790-741109e01fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:19.963986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82794240-5dd0-4b9f-b52d-7f0e37abddc2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:19.970324 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 209/407] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 06:16:19 2022] GET /identity/v3/groups/c19765ac480d491f889cab5cfb00ab26 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:19.978217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82794240-5dd0-4b9f-b52d-7f0e37abddc2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:20.011874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b803b36a-1ef9-47dd-b790-741109e01fd1 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:20.017183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b803b36a-1ef9-47dd-b790-741109e01fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:20.176992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 199/408] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:20.212350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15b749-a297-45ae-ae05-7d0b7a02307b None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:20.212987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15b749-a297-45ae-ae05-7d0b7a02307b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:20.213459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15b749-a297-45ae-ae05-7d0b7a02307b None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:20.215350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 200/409] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:20.216707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82794240-5dd0-4b9f-b52d-7f0e37abddc2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:20.218266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15b749-a297-45ae-ae05-7d0b7a02307b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:20.253915 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:20.284324 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd8d8a-a763-4d65-ae03-cdea8ba2b7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:20.286168 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd8d8a-a763-4d65-ae03-cdea8ba2b7dd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:20.286628 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd8d8a-a763-4d65-ae03-cdea8ba2b7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:20.287058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd8d8a-a763-4d65-ae03-cdea8ba2b7dd None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:20.289317 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd8d8a-a763-4d65-ae03-cdea8ba2b7dd None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:20.291599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd8d8a-a763-4d65-ae03-cdea8ba2b7dd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:20.294188 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 210/410] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:16:19 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:16:20.295834 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca412674-fadd-4768-916a-5524b9902a79 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:20.296295 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca412674-fadd-4768-916a-5524b9902a79 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:20.296751 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca412674-fadd-4768-916a-5524b9902a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:20.327189 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ca412674-fadd-4768-916a-5524b9902a79 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:20.329835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ca412674-fadd-4768-916a-5524b9902a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:20.395780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15b749-a297-45ae-ae05-7d0b7a02307b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:20.409245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:20.440290 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 211/411] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:20.441426 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca412674-fadd-4768-916a-5524b9902a79 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:20.468657 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859b074-b127-4364-ba8d-954ff56063c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:20.469827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859b074-b127-4364-ba8d-954ff56063c0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:20.470386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859b074-b127-4364-ba8d-954ff56063c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:20.470842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859b074-b127-4364-ba8d-954ff56063c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:20.486323 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5859b074-b127-4364-ba8d-954ff56063c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:20.496714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-5859b074-b127-4364-ba8d-954ff56063c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:16:20.496714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:20.496714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:20.496714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:20.496714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:20.496714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:20.496714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:20.496714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:20.496714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:20.496714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:20.496714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:20.496714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:20.496714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:20.496714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:20.496714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:20.496714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:20.497771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:20.497771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:20.497771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:20.497771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:20.497771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:20.497771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:20.497771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:20.497771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:20.497771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:20.497771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:20.497771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:20.497771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:20.497771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:20.497771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:20.497771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:20.497771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:20.498565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:20.498565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:20.498565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:20.498565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:20.498565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:20.498565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:20.498565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:20.498565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:20.498565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:20.498565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:20.498565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:16:20.498565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:20.501469 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 201/412] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:16:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:20.502550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859b074-b127-4364-ba8d-954ff56063c0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:20.614766 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca412674-fadd-4768-916a-5524b9902a79 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:20.632617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:20.657269 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859b074-b127-4364-ba8d-954ff56063c0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:20.665978 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee3aac-c6cc-4d3a-8d12-f14616bbda96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:20.667170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee3aac-c6cc-4d3a-8d12-f14616bbda96 None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:20.668318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee3aac-c6cc-4d3a-8d12-f14616bbda96 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:20.668810 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee3aac-c6cc-4d3a-8d12-f14616bbda96 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78/groups/c19765ac480d491f889cab5cfb00ab26/roles/6e5dce7d61b34704ad2cd32251fa4d34` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:20.672423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee3aac-c6cc-4d3a-8d12-f14616bbda96 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78, group_id=c19765ac480d491f889cab5cfb00ab26, role_id=6e5dce7d61b34704ad2cd32251fa4d34)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:20.674930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee3aac-c6cc-4d3a-8d12-f14616bbda96 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:20.700298 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 212/413] 10.222.0.65 () {58 vars in 1442 bytes} [Thu May 19 06:16:20 2022] PUT /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78/groups/c19765ac480d491f889cab5cfb00ab26/roles/6e5dce7d61b34704ad2cd32251fa4d34 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:16:20.700298 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee3aac-c6cc-4d3a-8d12-f14616bbda96 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:20.727511 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46eeca-700f-484b-96e5-daca7611c01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:20.731054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46eeca-700f-484b-96e5-daca7611c01f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:20.733718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46eeca-700f-484b-96e5-daca7611c01f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:20.734352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46eeca-700f-484b-96e5-daca7611c01f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:20.758422 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46eeca-700f-484b-96e5-daca7611c01f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:20.767359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-6e46eeca-700f-484b-96e5-daca7611c01f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:16:20.767359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:20.767359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:20.767359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:20.767359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:20.767359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:20.767359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:20.767359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:20.767359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:20.767359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:20.767359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:20.767359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:20.767359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:20.767359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:20.767359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:20.767359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:20.768312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:20.768312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:20.768312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:20.768312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:20.768312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:20.768312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:20.768312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:20.768312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:20.768312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:20.768312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:20.768312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:20.768312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:20.768312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:20.768312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:20.768312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:20.768312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:20.769130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:20.769130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:20.769130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:20.769130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:20.769130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:20.769130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:20.769130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:20.769130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:20.769130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:20.769130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:20.769130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:16:20.769130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:20.769130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 202/414] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:20.776592 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46eeca-700f-484b-96e5-daca7611c01f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:20.811639 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:20.998046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee3aac-c6cc-4d3a-8d12-f14616bbda96 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:21.002417 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46eeca-700f-484b-96e5-daca7611c01f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:21.049116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a595302-d02d-4f29-b990-d3485fd0830a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:21.049924 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a595302-d02d-4f29-b990-d3485fd0830a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:21.050213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a595302-d02d-4f29-b990-d3485fd0830a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:21.050525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a595302-d02d-4f29-b990-d3485fd0830a None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:21.051401 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a595302-d02d-4f29-b990-d3485fd0830a None admin] RBAC: Adding query filter params (name=member) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:21.051714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a595302-d02d-4f29-b990-d3485fd0830a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:21.053189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a595302-d02d-4f29-b990-d3485fd0830a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:21.072371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 203/415] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:16:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:21.073956 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556b592-c7ba-4d3e-a03e-9e23699bc048 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:21.074253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556b592-c7ba-4d3e-a03e-9e23699bc048 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:21.074535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556b592-c7ba-4d3e-a03e-9e23699bc048 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:21.075760 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121000b-0077-4645-98d7-ea9b39c87276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:21.076470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 204/416] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:21.076914 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121000b-0077-4645-98d7-ea9b39c87276 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:21.081226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121000b-0077-4645-98d7-ea9b39c87276 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:21.081226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121000b-0077-4645-98d7-ea9b39c87276 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:21.081226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1121000b-0077-4645-98d7-ea9b39c87276 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:21.081226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1121000b-0077-4645-98d7-ea9b39c87276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:21.085319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76687b1-576b-4963-a6bb-935af3d1973d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:21.085319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76687b1-576b-4963-a6bb-935af3d1973d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:21.085319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76687b1-576b-4963-a6bb-935af3d1973d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:21.085617 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1121000b-0077-4645-98d7-ea9b39c87276 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:21.100271 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d76687b1-576b-4963-a6bb-935af3d1973d None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:21.101537 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d76687b1-576b-4963-a6bb-935af3d1973d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:21.102033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 213/417] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:21.103895 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:21.104776 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121000b-0077-4645-98d7-ea9b39c87276 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:21.198286 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 205/418] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:21.199986 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76687b1-576b-4963-a6bb-935af3d1973d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:21.297411 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:21.302232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121000b-0077-4645-98d7-ea9b39c87276 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:21.349420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc955ba-87bc-439b-be26-97db6b1e364f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:21.350134 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc955ba-87bc-439b-be26-97db6b1e364f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:21.350428 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc955ba-87bc-439b-be26-97db6b1e364f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:21.350715 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc955ba-87bc-439b-be26-97db6b1e364f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:21.393477 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76687b1-576b-4963-a6bb-935af3d1973d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:21.454313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089114fa-0541-432a-8827-4353143ce06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:21.454885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc955ba-87bc-439b-be26-97db6b1e364f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:21.455731 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089114fa-0541-432a-8827-4353143ce06c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:21.456231 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089114fa-0541-432a-8827-4353143ce06c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:21.456406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc955ba-87bc-439b-be26-97db6b1e364f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:21.456725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089114fa-0541-432a-8827-4353143ce06c None admin] PATH_INFO: `/v3/groups/7f2fe75d30e9411aaced865d57ea3acc` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:21.459047 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089114fa-0541-432a-8827-4353143ce06c None admin] RBAC: Authorizing `identity:get_group(group_id=7f2fe75d30e9411aaced865d57ea3acc)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:21.461841 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089114fa-0541-432a-8827-4353143ce06c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:21.465500 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 206/419] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 06:16:21 2022] GET /identity/v3/groups/7f2fe75d30e9411aaced865d57ea3acc => generated 229 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:21.467149 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089114fa-0541-432a-8827-4353143ce06c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:21.533670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:21.573456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:21.585447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 214/420] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:21.586468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc955ba-87bc-439b-be26-97db6b1e364f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:21.639207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089114fa-0541-432a-8827-4353143ce06c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:21.678309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8dc11-116c-429d-9c5b-b27ec1810eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:21.679214 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8dc11-116c-429d-9c5b-b27ec1810eef None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:21.679501 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8dc11-116c-429d-9c5b-b27ec1810eef None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:21.679778 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8dc11-116c-429d-9c5b-b27ec1810eef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:21.692249 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8dc11-116c-429d-9c5b-b27ec1810eef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:21.701320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-2ad8dc11-116c-429d-9c5b-b27ec1810eef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:16:21.701320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:21.701320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:21.701320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:21.701320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:21.701320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:21.701320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:21.701320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:21.701320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:21.701320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:21.701320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:21.701320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:21.701320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:21.701320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:21.701320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:21.701320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:21.702444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:21.702444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:21.702444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:21.702444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:21.702444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:21.702444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:21.702444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:21.702444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:21.702444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:21.702444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:21.702444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:21.702444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:21.702444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:21.702444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:21.702444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:21.702444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:21.703722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:21.703722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:21.703722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:21.703722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:21.703722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:21.703722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:21.703722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:21.703722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:21.703722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:21.703722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:21.703722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:16:21.703722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:21.703722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 207/421] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:16:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:21.705014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8dc11-116c-429d-9c5b-b27ec1810eef None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:21.708706 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc955ba-87bc-439b-be26-97db6b1e364f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:21.766828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77c7df-c83d-4dd8-90d8-28ab22badaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:21.767709 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77c7df-c83d-4dd8-90d8-28ab22badaae None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:21.767994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77c7df-c83d-4dd8-90d8-28ab22badaae None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:21.768275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77c7df-c83d-4dd8-90d8-28ab22badaae None admin] PATH_INFO: `/v3/projects/0443648214584699a9ada0deda814964` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:21.769666 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77c7df-c83d-4dd8-90d8-28ab22badaae None admin] RBAC: Authorizing `identity:get_project(project_id=0443648214584699a9ada0deda814964)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:21.771159 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77c7df-c83d-4dd8-90d8-28ab22badaae None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:21.773236 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 215/422] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:16:21 2022] GET /identity/v3/projects/0443648214584699a9ada0deda814964 => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:21.774173 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77c7df-c83d-4dd8-90d8-28ab22badaae None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:21.783618 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:21.823979 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8dc11-116c-429d-9c5b-b27ec1810eef None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:21.850227 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:21.863787 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8535afb0-19b2-43f9-b75f-f8d60466c42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:21.864666 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535afb0-19b2-43f9-b75f-f8d60466c42b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:21.864952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535afb0-19b2-43f9-b75f-f8d60466c42b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:21.865230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535afb0-19b2-43f9-b75f-f8d60466c42b None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:21.866514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8535afb0-19b2-43f9-b75f-f8d60466c42b None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:21.867989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8535afb0-19b2-43f9-b75f-f8d60466c42b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:21.875288 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 208/423] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:16:21 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:21.875288 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8535afb0-19b2-43f9-b75f-f8d60466c42b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:21.898028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77c7df-c83d-4dd8-90d8-28ab22badaae None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:21.938864 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54d01a-e474-4872-b7b5-dc42741a299c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:21.939800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54d01a-e474-4872-b7b5-dc42741a299c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:21.940236 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54d01a-e474-4872-b7b5-dc42741a299c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:21.940683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54d01a-e474-4872-b7b5-dc42741a299c None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:21.941643 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54d01a-e474-4872-b7b5-dc42741a299c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:21.942140 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54d01a-e474-4872-b7b5-dc42741a299c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:21.943741 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54d01a-e474-4872-b7b5-dc42741a299c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:21.959335 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 216/424] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:21.960561 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54d01a-e474-4872-b7b5-dc42741a299c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:21.987815 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8535afb0-19b2-43f9-b75f-f8d60466c42b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:22.026019 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ca7c2-f0bb-4a3b-ae46-28fe000d31c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:22.026958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ca7c2-f0bb-4a3b-ae46-28fe000d31c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:22.027248 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ca7c2-f0bb-4a3b-ae46-28fe000d31c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:22.027526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ca7c2-f0bb-4a3b-ae46-28fe000d31c0 None admin] PATH_INFO: `/v3/projects/0443648214584699a9ada0deda814964/groups/7f2fe75d30e9411aaced865d57ea3acc/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:22.029755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ca7c2-f0bb-4a3b-ae46-28fe000d31c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=0443648214584699a9ada0deda814964, group_id=7f2fe75d30e9411aaced865d57ea3acc, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:22.031583 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ca7c2-f0bb-4a3b-ae46-28fe000d31c0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:22.034144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:22.052460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 209/425] 10.222.0.65 () {58 vars in 1442 bytes} [Thu May 19 06:16:21 2022] PUT /identity/v3/projects/0443648214584699a9ada0deda814964/groups/7f2fe75d30e9411aaced865d57ea3acc/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:16:22.053756 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ca7c2-f0bb-4a3b-ae46-28fe000d31c0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:22.088301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54d01a-e474-4872-b7b5-dc42741a299c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:22.134719 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:22.160721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b794014-ff32-4ddc-8a85-26fa17113795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:22.161643 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b794014-ff32-4ddc-8a85-26fa17113795 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:22.161932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b794014-ff32-4ddc-8a85-26fa17113795 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:22.162219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b794014-ff32-4ddc-8a85-26fa17113795 None admin] PATH_INFO: `/v3/groups/c19765ac480d491f889cab5cfb00ab26` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:22.166211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b794014-ff32-4ddc-8a85-26fa17113795 None admin] RBAC: Authorizing `identity:get_group(group_id=c19765ac480d491f889cab5cfb00ab26)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:22.167765 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b794014-ff32-4ddc-8a85-26fa17113795 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:22.170439 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 217/426] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 06:16:21 2022] GET /identity/v3/groups/c19765ac480d491f889cab5cfb00ab26 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:22.176444 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b794014-ff32-4ddc-8a85-26fa17113795 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:22.189133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ca7c2-f0bb-4a3b-ae46-28fe000d31c0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:22.233641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce3476-9108-48e3-9320-948f117a69d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:22.234573 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce3476-9108-48e3-9320-948f117a69d2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:22.235009 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce3476-9108-48e3-9320-948f117a69d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:22.235436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce3476-9108-48e3-9320-948f117a69d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:22.326452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:22.348103 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b794014-ff32-4ddc-8a85-26fa17113795 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:22.348988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce3476-9108-48e3-9320-948f117a69d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:22.350572 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce3476-9108-48e3-9320-948f117a69d2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:22.392593 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abeffb-ce63-4398-ba23-24c356645627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:22.393308 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abeffb-ce63-4398-ba23-24c356645627 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:22.393635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abeffb-ce63-4398-ba23-24c356645627 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:22.393922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abeffb-ce63-4398-ba23-24c356645627 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:22.394804 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93abeffb-ce63-4398-ba23-24c356645627 None admin] RBAC: Adding query filter params (group.id=c19765ac480d491f889cab5cfb00ab26, role.id=ed6cf002b70f422190946ebcb114da60, scope.project.id=fda8b64de40a4fd698a2458618d1a582) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:22.395126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93abeffb-ce63-4398-ba23-24c356645627 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:22.396673 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93abeffb-ce63-4398-ba23-24c356645627 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:22.405532 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 218/427] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 06:16:22 2022] GET /identity/v3/role_assignments?group.id=c19765ac480d491f889cab5cfb00ab26&scope.project.id=fda8b64de40a4fd698a2458618d1a582&role.id=ed6cf002b70f422190946ebcb114da60 => generated 611 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:16:22.510071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 210/428] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:22.516885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abeffb-ce63-4398-ba23-24c356645627 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:22.605679 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:22.675285 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abeffb-ce63-4398-ba23-24c356645627 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:22.728591 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742f28d-0fa7-437d-88cc-defe82b5629d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:22.729536 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742f28d-0fa7-437d-88cc-defe82b5629d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:22.729826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742f28d-0fa7-437d-88cc-defe82b5629d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:22.730114 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742f28d-0fa7-437d-88cc-defe82b5629d None admin] PATH_INFO: `/v3/projects/fda8b64de40a4fd698a2458618d1a582` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:22.734241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742f28d-0fa7-437d-88cc-defe82b5629d None admin] RBAC: Authorizing `identity:get_project(project_id=fda8b64de40a4fd698a2458618d1a582)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:22.735787 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742f28d-0fa7-437d-88cc-defe82b5629d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:22.738101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 219/429] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:16:22 2022] GET /identity/v3/projects/fda8b64de40a4fd698a2458618d1a582 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:22.747761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce3476-9108-48e3-9320-948f117a69d2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:22.833172 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238a5a1-33f2-43d3-9ee6-6704967970b5 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:22.833535 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238a5a1-33f2-43d3-9ee6-6704967970b5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:22.833766 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238a5a1-33f2-43d3-9ee6-6704967970b5 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:22.835323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 220/430] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:16:22.841621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb21c6-4a5b-4013-8ac2-2fa988524142 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:22.842262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb21c6-4a5b-4013-8ac2-2fa988524142 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:22.842796 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb21c6-4a5b-4013-8ac2-2fa988524142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:22.884281 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e1fb21c6-4a5b-4013-8ac2-2fa988524142 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:22.887045 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e1fb21c6-4a5b-4013-8ac2-2fa988524142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:22.909720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:22.918117 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce3476-9108-48e3-9320-948f117a69d2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:22.967190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02b91d-8a32-44f7-8466-3ec2d8a933d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:22.969119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02b91d-8a32-44f7-8466-3ec2d8a933d9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:22.969928 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02b91d-8a32-44f7-8466-3ec2d8a933d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:22.970659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02b91d-8a32-44f7-8466-3ec2d8a933d9 None admin] PATH_INFO: `/v3/groups/c19765ac480d491f889cab5cfb00ab26` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:22.973532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02b91d-8a32-44f7-8466-3ec2d8a933d9 None admin] RBAC: Authorizing `identity:get_group(group_id=c19765ac480d491f889cab5cfb00ab26)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:22.976701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02b91d-8a32-44f7-8466-3ec2d8a933d9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:22.980730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 211/431] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 06:16:22 2022] GET /identity/v3/groups/c19765ac480d491f889cab5cfb00ab26 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:16:22.991553 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02b91d-8a32-44f7-8466-3ec2d8a933d9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:23.048997 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 221/432] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:16:23.099583 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb21c6-4a5b-4013-8ac2-2fa988524142 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:23.207075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02b91d-8a32-44f7-8466-3ec2d8a933d9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:23.243067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:23.282293 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce4959-2a4e-455f-85e0-8b401e589ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:23.283985 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce4959-2a4e-455f-85e0-8b401e589ade None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:23.284718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce4959-2a4e-455f-85e0-8b401e589ade None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:23.285558 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce4959-2a4e-455f-85e0-8b401e589ade None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:23.287474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce4959-2a4e-455f-85e0-8b401e589ade None admin] RBAC: Adding query filter params (group.id=c19765ac480d491f889cab5cfb00ab26, role.id=6e5dce7d61b34704ad2cd32251fa4d34, scope.project.id=fda8b64de40a4fd698a2458618d1a582) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:23.288296 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce4959-2a4e-455f-85e0-8b401e589ade None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:23.291294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce4959-2a4e-455f-85e0-8b401e589ade None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:23.306570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 212/433] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 06:16:22 2022] GET /identity/v3/role_assignments?group.id=c19765ac480d491f889cab5cfb00ab26&scope.project.id=fda8b64de40a4fd698a2458618d1a582&role.id=6e5dce7d61b34704ad2cd32251fa4d34 => generated 611 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:23.309346 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ce9ec-8b0e-42a1-9fb9-8731d6ddc299 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:23.315735 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ce9ec-8b0e-42a1-9fb9-8731d6ddc299 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:23.317060 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ce9ec-8b0e-42a1-9fb9-8731d6ddc299 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:23.321374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 213/434] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:23 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:23.330270 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c25cf-d74d-437f-bf8a-07c6953c339e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:23.331310 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c25cf-d74d-437f-bf8a-07c6953c339e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:23.331310 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c25cf-d74d-437f-bf8a-07c6953c339e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:23.337214 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb21c6-4a5b-4013-8ac2-2fa988524142 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:23.355416 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4a5c25cf-d74d-437f-bf8a-07c6953c339e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:23.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4a5c25cf-d74d-437f-bf8a-07c6953c339e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:23.382933 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:23.421767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5510212-cd7e-45ce-89fe-6a6c88d32ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:23.423826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5510212-cd7e-45ce-89fe-6a6c88d32ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:23.424698 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5510212-cd7e-45ce-89fe-6a6c88d32ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:23.425557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5510212-cd7e-45ce-89fe-6a6c88d32ce3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:23.448593 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5510212-cd7e-45ce-89fe-6a6c88d32ce3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:23.449501 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 214/435] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:23.466469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-c5510212-cd7e-45ce-89fe-6a6c88d32ce3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 06:16:23.466469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:23.466469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:23.466469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:23.466469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:23.466469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:23.466469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:23.466469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:23.466469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:23.466469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:23.466469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:23.466469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:23.466469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:23.466469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:23.466469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:23.466469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:23.468266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:23.468266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:23.468266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:23.468266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:23.468266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:23.468266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:23.468266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:23.468266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:23.468266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:23.468266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:23.468266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:23.468266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:23.468266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:23.468266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:23.468266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:23.468266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:23.469990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:23.469990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:23.469990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:23.469990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:23.469990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:23.469990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:23.469990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:23.469990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:23.469990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:23.469990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:23.469990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 06:16:23.469990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:23.471190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c25cf-d74d-437f-bf8a-07c6953c339e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:23.475329 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 222/436] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:23.486251 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5510212-cd7e-45ce-89fe-6a6c88d32ce3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:23.620255 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c25cf-d74d-437f-bf8a-07c6953c339e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:23.664312 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5510212-cd7e-45ce-89fe-6a6c88d32ce3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:23.671924 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a5056-5631-4a95-b5fd-0564373e6228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:23.673024 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a5056-5631-4a95-b5fd-0564373e6228 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:23.673515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a5056-5631-4a95-b5fd-0564373e6228 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:23.673938 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a5056-5631-4a95-b5fd-0564373e6228 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:23.685650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:23.688225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a5056-5631-4a95-b5fd-0564373e6228 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:23.697459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-6a3a5056-5631-4a95-b5fd-0564373e6228 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:16:23.697459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:23.697459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:23.697459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:23.697459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:23.697459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:23.697459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:23.697459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:23.697459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:23.697459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:23.697459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:23.697459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:23.697459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:23.697459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:23.697459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:23.697459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:23.698551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:23.698551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:23.698551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:23.698551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:23.698551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:23.698551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:23.698551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:23.698551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:23.698551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:23.698551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:23.698551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:23.698551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:23.698551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:23.698551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:23.698551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:23.698551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:23.699447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:23.699447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:23.699447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:23.699447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:23.699447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:23.699447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:23.699447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:23.699447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:23.699447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:23.699447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:23.699447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:16:23.699447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:23.702256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 215/437] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:16:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:16:23.708565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a5056-5631-4a95-b5fd-0564373e6228 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:23.714762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9b8f9-d86f-429e-a05a-a560d3d7a14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:23.715459 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9b8f9-d86f-429e-a05a-a560d3d7a14f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:23.716333 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9b8f9-d86f-429e-a05a-a560d3d7a14f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:23.716686 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9b8f9-d86f-429e-a05a-a560d3d7a14f None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:23.717466 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9b8f9-d86f-429e-a05a-a560d3d7a14f None admin] RBAC: Adding query filter params (name=member) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:23.717784 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9b8f9-d86f-429e-a05a-a560d3d7a14f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:23.719226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9b8f9-d86f-429e-a05a-a560d3d7a14f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:23.730497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 223/438] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:16:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:23.740485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9b8f9-d86f-429e-a05a-a560d3d7a14f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:23.742724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:23.869872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a5056-5631-4a95-b5fd-0564373e6228 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:23.937260 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83409a-bfaf-4aa2-89ec-75307c245eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:23.938262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83409a-bfaf-4aa2-89ec-75307c245eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:23.938262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83409a-bfaf-4aa2-89ec-75307c245eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:23.938571 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83409a-bfaf-4aa2-89ec-75307c245eb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:23.939316 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83409a-bfaf-4aa2-89ec-75307c245eb7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:23.939630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83409a-bfaf-4aa2-89ec-75307c245eb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:23.941084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83409a-bfaf-4aa2-89ec-75307c245eb7 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:23.953456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 216/439] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:16:23.960543 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9b8f9-d86f-429e-a05a-a560d3d7a14f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:23.963705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83409a-bfaf-4aa2-89ec-75307c245eb7 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:24.019119 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab3157-1a4b-4a92-a31b-540a568686a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:24.020117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab3157-1a4b-4a92-a31b-540a568686a2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:24.020622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab3157-1a4b-4a92-a31b-540a568686a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:24.021125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab3157-1a4b-4a92-a31b-540a568686a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:24.109107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:24.116059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:24.149155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83409a-bfaf-4aa2-89ec-75307c245eb7 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:24.157150 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab3157-1a4b-4a92-a31b-540a568686a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:24.159962 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab3157-1a4b-4a92-a31b-540a568686a2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:24.211599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba87b7-e90f-4fa9-b64d-06595360a8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:24.212732 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba87b7-e90f-4fa9-b64d-06595360a8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:24.213331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba87b7-e90f-4fa9-b64d-06595360a8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:24.213847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba87b7-e90f-4fa9-b64d-06595360a8d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:24.333430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba87b7-e90f-4fa9-b64d-06595360a8d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:24.334832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba87b7-e90f-4fa9-b64d-06595360a8d3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:24.337677 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 224/440] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:24.341667 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab3157-1a4b-4a92-a31b-540a568686a2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:24.484425 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:24.485977 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:24.519111 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab3157-1a4b-4a92-a31b-540a568686a2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:24.530990 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 217/441] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 06:16:24.539454 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba87b7-e90f-4fa9-b64d-06595360a8d3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:24.588568 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794aecd8-1d5e-4f43-b125-a5bf4830f48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:24.589732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794aecd8-1d5e-4f43-b125-a5bf4830f48c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:24.590188 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794aecd8-1d5e-4f43-b125-a5bf4830f48c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:24.590693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794aecd8-1d5e-4f43-b125-a5bf4830f48c None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:24.592831 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794aecd8-1d5e-4f43-b125-a5bf4830f48c None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:24.595363 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794aecd8-1d5e-4f43-b125-a5bf4830f48c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:24.598242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 225/442] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:16:24 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:16:24.607081 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794aecd8-1d5e-4f43-b125-a5bf4830f48c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:24.724557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba87b7-e90f-4fa9-b64d-06595360a8d3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:24.784972 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d74d8-6b80-4ad2-a939-28c5f5231230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:24.785206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794aecd8-1d5e-4f43-b125-a5bf4830f48c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:24.786532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d74d8-6b80-4ad2-a939-28c5f5231230 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:24.786990 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d74d8-6b80-4ad2-a939-28c5f5231230 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:24.787440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d74d8-6b80-4ad2-a939-28c5f5231230 None admin] PATH_INFO: `/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:24.789675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d74d8-6b80-4ad2-a939-28c5f5231230 None admin] RBAC: Authorizing `identity:get_project(project_id=5c0f7ddae25d4ddd8ef1e2d8e5429d78)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:24.792004 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d74d8-6b80-4ad2-a939-28c5f5231230 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:24.796414 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 218/443] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:16:24 2022] GET /identity/v3/projects/5c0f7ddae25d4ddd8ef1e2d8e5429d78 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:16:24.804659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d74d8-6b80-4ad2-a939-28c5f5231230 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:24.848686 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e5b64-1555-4cee-9cd2-3e3812b80c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:24.850006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e5b64-1555-4cee-9cd2-3e3812b80c4d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:24.850480 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e5b64-1555-4cee-9cd2-3e3812b80c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:24.850907 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e5b64-1555-4cee-9cd2-3e3812b80c4d None admin] PATH_INFO: `/v3/groups/c19765ac480d491f889cab5cfb00ab26` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:24.852914 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e5b64-1555-4cee-9cd2-3e3812b80c4d None admin] RBAC: Authorizing `identity:get_group(group_id=c19765ac480d491f889cab5cfb00ab26)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:24.855431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e5b64-1555-4cee-9cd2-3e3812b80c4d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:24.859054 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 226/444] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 06:16:24 2022] GET /identity/v3/groups/c19765ac480d491f889cab5cfb00ab26 => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:24.865533 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e5b64-1555-4cee-9cd2-3e3812b80c4d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:24.880601 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:24.894030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:24.982097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d74d8-6b80-4ad2-a939-28c5f5231230 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:25.040274 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e5b64-1555-4cee-9cd2-3e3812b80c4d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:25.042428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f81a65-f999-4ea6-9b83-10a26342d8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:25.044373 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f81a65-f999-4ea6-9b83-10a26342d8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:25.044846 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f81a65-f999-4ea6-9b83-10a26342d8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:25.045181 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f81a65-f999-4ea6-9b83-10a26342d8f7 None admin] PATH_INFO: `/v3/groups/c19765ac480d491f889cab5cfb00ab26` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:25.046973 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f81a65-f999-4ea6-9b83-10a26342d8f7 None admin] RBAC: Authorizing `identity:get_group(group_id=c19765ac480d491f889cab5cfb00ab26)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:25.049571 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f81a65-f999-4ea6-9b83-10a26342d8f7 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:25.057237 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 219/445] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 06:16:24 2022] GET /identity/v3/groups/c19765ac480d491f889cab5cfb00ab26 => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:25.058575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f81a65-f999-4ea6-9b83-10a26342d8f7 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:25.102919 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2332ed-ec5b-40f2-867c-eb9590583d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:25.103500 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2332ed-ec5b-40f2-867c-eb9590583d07 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:25.103790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2332ed-ec5b-40f2-867c-eb9590583d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:25.104075 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2332ed-ec5b-40f2-867c-eb9590583d07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:25.104987 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2332ed-ec5b-40f2-867c-eb9590583d07 None admin] RBAC: Adding query filter params (group.id=c19765ac480d491f889cab5cfb00ab26, role.id=ed6cf002b70f422190946ebcb114da60, scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:25.105345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2332ed-ec5b-40f2-867c-eb9590583d07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:25.107221 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2332ed-ec5b-40f2-867c-eb9590583d07 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:25.115269 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 227/446] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 06:16:24 2022] GET /identity/v3/role_assignments?group.id=c19765ac480d491f889cab5cfb00ab26&scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78&role.id=ed6cf002b70f422190946ebcb114da60 => generated 611 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:25.118049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906efaf-44dc-45c7-8bc6-fb098494efec None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:25.118659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906efaf-44dc-45c7-8bc6-fb098494efec None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:25.118975 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906efaf-44dc-45c7-8bc6-fb098494efec None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:25.122147 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 228/447] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:25.130739 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab76ba3-fa2e-42b5-8fe9-eeeb529a48ee None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:25.131180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab76ba3-fa2e-42b5-8fe9-eeeb529a48ee None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:25.131624 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab76ba3-fa2e-42b5-8fe9-eeeb529a48ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:25.161854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-4ab76ba3-fa2e-42b5-8fe9-eeeb529a48ee None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:25.163761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4ab76ba3-fa2e-42b5-8fe9-eeeb529a48ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:25.234005 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f81a65-f999-4ea6-9b83-10a26342d8f7 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:25.256205 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:25.270732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 229/448] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:16:25.285427 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a73c2-a1b3-4c8e-bba3-3858eb411c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:25.286320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a73c2-a1b3-4c8e-bba3-3858eb411c7c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:25.286789 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a73c2-a1b3-4c8e-bba3-3858eb411c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:25.287182 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a73c2-a1b3-4c8e-bba3-3858eb411c7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:25.288058 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a73c2-a1b3-4c8e-bba3-3858eb411c7c None admin] RBAC: Adding query filter params (group.id=c19765ac480d491f889cab5cfb00ab26, role.id=6e5dce7d61b34704ad2cd32251fa4d34, scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:25.288383 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a73c2-a1b3-4c8e-bba3-3858eb411c7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:25.289909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a73c2-a1b3-4c8e-bba3-3858eb411c7c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:25.298123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 220/449] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 06:16:25 2022] GET /identity/v3/role_assignments?group.id=c19765ac480d491f889cab5cfb00ab26&scope.project.id=5c0f7ddae25d4ddd8ef1e2d8e5429d78&role.id=6e5dce7d61b34704ad2cd32251fa4d34 => generated 611 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:16:25.314634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab76ba3-fa2e-42b5-8fe9-eeeb529a48ee None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:25.349718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:25.493406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab76ba3-fa2e-42b5-8fe9-eeeb529a48ee None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:25.546826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40bcfd-2eb3-4800-b9bb-6557106ae586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:25.547685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40bcfd-2eb3-4800-b9bb-6557106ae586 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:25.547973 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40bcfd-2eb3-4800-b9bb-6557106ae586 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:25.548253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40bcfd-2eb3-4800-b9bb-6557106ae586 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:25.563126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40bcfd-2eb3-4800-b9bb-6557106ae586 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:25.571399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-7e40bcfd-2eb3-4800-b9bb-6557106ae586 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:16:25.571399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:25.571399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:25.571399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:25.571399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:25.571399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:25.571399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:25.571399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:25.571399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:25.571399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:25.571399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:25.571399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:25.571399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:25.571399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:25.571399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:25.571399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:25.572934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:25.572934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:25.572934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:25.572934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:25.572934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:25.572934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:25.572934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:25.572934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:25.572934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:25.572934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:25.572934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:25.572934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:25.572934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:25.572934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:25.572934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:25.572934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:25.574384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:25.574384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:25.574384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:25.574384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:25.574384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:25.574384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:25.574384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:25.574384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:25.574384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:25.574384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:25.574384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:16:25.574384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:25.574384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 230/450] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:16:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:25.583557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a73c2-a1b3-4c8e-bba3-3858eb411c7c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:25.771150 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a73c2-a1b3-4c8e-bba3-3858eb411c7c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:25.836758 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc7498-767c-443c-a884-6bd31edc3bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:25.837947 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc7498-767c-443c-a884-6bd31edc3bcc None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:25.838447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc7498-767c-443c-a884-6bd31edc3bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:25.838947 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc7498-767c-443c-a884-6bd31edc3bcc None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:25.840378 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc7498-767c-443c-a884-6bd31edc3bcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:25.841465 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc7498-767c-443c-a884-6bd31edc3bcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:25.843664 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc7498-767c-443c-a884-6bd31edc3bcc None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:25.859667 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 221/451] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:25.867990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40bcfd-2eb3-4800-b9bb-6557106ae586 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:25.953976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:25.993081 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40bcfd-2eb3-4800-b9bb-6557106ae586 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:26.033226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658814ac-cd6e-46d7-a534-ad29f04c44da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:26.033844 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658814ac-cd6e-46d7-a534-ad29f04c44da None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:26.034135 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658814ac-cd6e-46d7-a534-ad29f04c44da None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:26.034412 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658814ac-cd6e-46d7-a534-ad29f04c44da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:26.112192 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658814ac-cd6e-46d7-a534-ad29f04c44da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:26.113735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658814ac-cd6e-46d7-a534-ad29f04c44da None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:26.191582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:26.232626 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 231/452] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:26.238760 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc7498-767c-443c-a884-6bd31edc3bcc None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:26.290205 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:26.361737 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc7498-767c-443c-a884-6bd31edc3bcc None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:26.401796 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637190f-9ce2-4223-8394-59d77d91fb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:26.402702 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637190f-9ce2-4223-8394-59d77d91fb5a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:26.403385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637190f-9ce2-4223-8394-59d77d91fb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:26.403385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637190f-9ce2-4223-8394-59d77d91fb5a None admin] PATH_INFO: `/v3/projects/0443648214584699a9ada0deda814964` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:26.405497 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637190f-9ce2-4223-8394-59d77d91fb5a None admin] RBAC: Authorizing `identity:get_project(project_id=0443648214584699a9ada0deda814964)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:26.407015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637190f-9ce2-4223-8394-59d77d91fb5a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:26.408933 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 222/453] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:16:26 2022] GET /identity/v3/projects/0443648214584699a9ada0deda814964 => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:26.414723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658814ac-cd6e-46d7-a534-ad29f04c44da None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:26.526283 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658814ac-cd6e-46d7-a534-ad29f04c44da None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:26.561971 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f63e2e-96da-4dea-a78a-f0ac5ca2d046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:26.562790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f63e2e-96da-4dea-a78a-f0ac5ca2d046 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:26.563117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f63e2e-96da-4dea-a78a-f0ac5ca2d046 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:26.563437 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f63e2e-96da-4dea-a78a-f0ac5ca2d046 None admin] PATH_INFO: `/v3/groups/7f2fe75d30e9411aaced865d57ea3acc` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:26.564709 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f63e2e-96da-4dea-a78a-f0ac5ca2d046 None admin] RBAC: Authorizing `identity:get_group(group_id=7f2fe75d30e9411aaced865d57ea3acc)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:26.566517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f63e2e-96da-4dea-a78a-f0ac5ca2d046 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:26.568324 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 232/454] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 06:16:26 2022] GET /identity/v3/groups/7f2fe75d30e9411aaced865d57ea3acc => generated 229 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:26.574322 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637190f-9ce2-4223-8394-59d77d91fb5a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:26.678661 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637190f-9ce2-4223-8394-59d77d91fb5a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:26.688729 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:26.715496 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb8dec-f07c-4403-8f59-beb40f5850f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:26.716833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb8dec-f07c-4403-8f59-beb40f5850f2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:26.717324 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb8dec-f07c-4403-8f59-beb40f5850f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:26.717613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb8dec-f07c-4403-8f59-beb40f5850f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:26.718457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb8dec-f07c-4403-8f59-beb40f5850f2 None admin] RBAC: Adding query filter params (group.id=7f2fe75d30e9411aaced865d57ea3acc, role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=0443648214584699a9ada0deda814964) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:26.718774 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb8dec-f07c-4403-8f59-beb40f5850f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:26.720248 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb8dec-f07c-4403-8f59-beb40f5850f2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:26.726715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 223/455] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 06:16:26 2022] GET /identity/v3/role_assignments?group.id=7f2fe75d30e9411aaced865d57ea3acc&scope.project.id=0443648214584699a9ada0deda814964&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:29.488900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b979f-65e2-44b6-bf79-f8ae7c643e3a None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:29.489820 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b979f-65e2-44b6-bf79-f8ae7c643e3a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:29.490259 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b979f-65e2-44b6-bf79-f8ae7c643e3a None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:29.492117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 233/456] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:29.499223 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5cbe7-8714-4b8c-8581-d15981dd1f21 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:29.499716 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5cbe7-8714-4b8c-8581-d15981dd1f21 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:29.500189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5cbe7-8714-4b8c-8581-d15981dd1f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:29.531822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-76f5cbe7-8714-4b8c-8581-d15981dd1f21 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:29.533370 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-76f5cbe7-8714-4b8c-8581-d15981dd1f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:29.643192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 224/457] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:29.686847 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b979f-65e2-44b6-bf79-f8ae7c643e3a None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:29.753232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:29.883953 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b979f-65e2-44b6-bf79-f8ae7c643e3a None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:29.963930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d429f4-4c55-40fb-a799-41f86313ac41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:29.965169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d429f4-4c55-40fb-a799-41f86313ac41 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:29.966198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d429f4-4c55-40fb-a799-41f86313ac41 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:29.966688 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d429f4-4c55-40fb-a799-41f86313ac41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:29.991092 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d429f4-4c55-40fb-a799-41f86313ac41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:29.998347 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d429f4-4c55-40fb-a799-41f86313ac41 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:30.041872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-35d429f4-4c55-40fb-a799-41f86313ac41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:30.041872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:30.041872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:30.041872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:30.041872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:30.041872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:30.041872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:30.041872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:30.041872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:30.041872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:30.041872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:30.041872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:30.041872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:30.041872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:30.041872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:30.044096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:30.044096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:30.044096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:30.044096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:30.044096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:30.044096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:30.044096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:30.044096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:30.044096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:30.044096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:30.044096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:30.044096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:30.044096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:30.044096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:30.044096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:30.044096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:30.044096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:30.045750 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:30.045750 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:30.045750 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:30.045750 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:30.045750 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 234/458] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:16:30.050675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5cbe7-8714-4b8c-8581-d15981dd1f21 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:30.265343 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5cbe7-8714-4b8c-8581-d15981dd1f21 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:30.267764 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:30.364844 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891d1d7-080e-4ab1-84db-a6d9d88a39e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:30.366529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891d1d7-080e-4ab1-84db-a6d9d88a39e7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:30.367164 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891d1d7-080e-4ab1-84db-a6d9d88a39e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:30.367917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891d1d7-080e-4ab1-84db-a6d9d88a39e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:30.369638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891d1d7-080e-4ab1-84db-a6d9d88a39e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:30.370519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891d1d7-080e-4ab1-84db-a6d9d88a39e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:30.373958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891d1d7-080e-4ab1-84db-a6d9d88a39e7 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:30.407740 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 225/459] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:30.416477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d429f4-4c55-40fb-a799-41f86313ac41 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:30.541966 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cec49-f587-46ec-b285-88a66035141c None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:30.543497 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cec49-f587-46ec-b285-88a66035141c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:30.544401 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cec49-f587-46ec-b285-88a66035141c None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:30.547526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 226/460] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:30.553740 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9222bc-18d3-4416-930b-61f2ab5549f4 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:30.554432 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9222bc-18d3-4416-930b-61f2ab5549f4 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:30.554925 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9222bc-18d3-4416-930b-61f2ab5549f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:30.584777 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-fa9222bc-18d3-4416-930b-61f2ab5549f4 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:30.586039 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fa9222bc-18d3-4416-930b-61f2ab5549f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:30.661671 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:30.682715 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d429f4-4c55-40fb-a799-41f86313ac41 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:30.725205 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 227/461] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:16:30.750415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9222bc-18d3-4416-930b-61f2ab5549f4 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:30.764577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f8a72-5ec1-4d36-bf74-952b16ff2519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:30.765734 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f8a72-5ec1-4d36-bf74-952b16ff2519 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:30.766220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f8a72-5ec1-4d36-bf74-952b16ff2519 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:30.766787 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f8a72-5ec1-4d36-bf74-952b16ff2519 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:30.767851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f8a72-5ec1-4d36-bf74-952b16ff2519 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:30.769687 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f8a72-5ec1-4d36-bf74-952b16ff2519 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:30.796770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 235/462] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:16:30 2022] POST /identity/v3/users => generated 246 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:16:30.799301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f7d2c-3dda-4632-96fd-091344bc10ca None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:30.799871 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f7d2c-3dda-4632-96fd-091344bc10ca None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:30.800426 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f7d2c-3dda-4632-96fd-091344bc10ca None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:30.807837 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 236/463] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:30.810169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8070b8-0968-49c3-a283-498ddf806861 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:30.810459 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8070b8-0968-49c3-a283-498ddf806861 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:30.810742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8070b8-0968-49c3-a283-498ddf806861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:30.829637 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5d8070b8-0968-49c3-a283-498ddf806861 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:30.830864 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5d8070b8-0968-49c3-a283-498ddf806861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:30.878323 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:30.892809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9222bc-18d3-4416-930b-61f2ab5549f4 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:30.914982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 237/464] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:16:30.945842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8070b8-0968-49c3-a283-498ddf806861 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:30.982732 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ab67a-6f83-433b-8b80-ead6ba01d25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:30.983583 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ab67a-6f83-433b-8b80-ead6ba01d25a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:30.983874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ab67a-6f83-433b-8b80-ead6ba01d25a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:30.984156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ab67a-6f83-433b-8b80-ead6ba01d25a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:31.003971 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ab67a-6f83-433b-8b80-ead6ba01d25a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:31.005573 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ab67a-6f83-433b-8b80-ead6ba01d25a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:31.052042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-745ab67a-6f83-433b-8b80-ead6ba01d25a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:31.052042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:31.052042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:31.052042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:31.052042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:31.052042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:31.052042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:31.052042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:31.052042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:31.052042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:31.052042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:31.052042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:31.052042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:31.052042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:31.052042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:31.052921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:31.052921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:31.052921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:31.052921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:31.052921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:31.052921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:31.052921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:31.052921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:31.052921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:31.052921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:31.052921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:31.052921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:31.052921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:31.052921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:31.052921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:31.052921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:31.052921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:31.053809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:31.053809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:31.053809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:31.053809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:31.053809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 228/465] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:16:31.063858 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ab67a-6f83-433b-8b80-ead6ba01d25a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:31.079932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:31.128778 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8070b8-0968-49c3-a283-498ddf806861 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:31.180373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bce46a-4c5f-4b8b-9606-dd85fb561c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:31.181183 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bce46a-4c5f-4b8b-9606-dd85fb561c7d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:31.181511 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bce46a-4c5f-4b8b-9606-dd85fb561c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:31.181790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bce46a-4c5f-4b8b-9606-dd85fb561c7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:31.193188 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bce46a-4c5f-4b8b-9606-dd85fb561c7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:31.194865 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bce46a-4c5f-4b8b-9606-dd85fb561c7d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:31.203077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ab67a-6f83-433b-8b80-ead6ba01d25a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:31.214792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-35bce46a-4c5f-4b8b-9606-dd85fb561c7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:31.214792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:31.214792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:31.214792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:31.214792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:31.214792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:31.214792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:31.214792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:31.214792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:31.214792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:31.214792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:31.214792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:31.214792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:31.214792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:31.214792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:31.215791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:31.215791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:31.215791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:31.215791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:31.215791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:31.215791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:31.215791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:31.215791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:31.215791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:31.215791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:31.215791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:31.215791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:31.215791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:31.215791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:31.215791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:31.215791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:31.215791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:31.216929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:31.216929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:31.216929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:31.216929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:31.217652 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 238/466] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:16:31.218746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d5a96-2679-4ffd-8adb-ebba172370a7 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:31.219043 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d5a96-2679-4ffd-8adb-ebba172370a7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:31.219326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d5a96-2679-4ffd-8adb-ebba172370a7 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:31.220900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 239/467] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:31.231698 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6518dd2-2b89-4587-b24f-86956f0d347b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:31.231698 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6518dd2-2b89-4587-b24f-86956f0d347b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:31.231698 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6518dd2-2b89-4587-b24f-86956f0d347b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:31.255744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21dd73-5b45-4b82-811e-35f28cb7c255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:31.256529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21dd73-5b45-4b82-811e-35f28cb7c255 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:31.256800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21dd73-5b45-4b82-811e-35f28cb7c255 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:31.257094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21dd73-5b45-4b82-811e-35f28cb7c255 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:31.257437 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e6518dd2-2b89-4587-b24f-86956f0d347b None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:31.257945 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21dd73-5b45-4b82-811e-35f28cb7c255 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:31.258264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21dd73-5b45-4b82-811e-35f28cb7c255 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:31.258721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e6518dd2-2b89-4587-b24f-86956f0d347b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:31.259731 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21dd73-5b45-4b82-811e-35f28cb7c255 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:31.275696 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 229/468] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:31.276622 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21dd73-5b45-4b82-811e-35f28cb7c255 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:31.294362 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:31.380720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 240/469] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:31.382443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6518dd2-2b89-4587-b24f-86956f0d347b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:31.479345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:31.501358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21dd73-5b45-4b82-811e-35f28cb7c255 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:31.540433 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48951c7-edfa-4cdf-810f-94e20dbb4b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:31.541122 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48951c7-edfa-4cdf-810f-94e20dbb4b62 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:31.541464 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48951c7-edfa-4cdf-810f-94e20dbb4b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:31.541747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48951c7-edfa-4cdf-810f-94e20dbb4b62 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:31.542497 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48951c7-edfa-4cdf-810f-94e20dbb4b62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:31.543023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48951c7-edfa-4cdf-810f-94e20dbb4b62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:31.545420 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48951c7-edfa-4cdf-810f-94e20dbb4b62 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:31.557984 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 230/470] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:31.558856 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48951c7-edfa-4cdf-810f-94e20dbb4b62 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:31.580356 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6518dd2-2b89-4587-b24f-86956f0d347b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:31.654534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f2168-3a2a-4c84-a805-958e3b02cb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:31.656131 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f2168-3a2a-4c84-a805-958e3b02cb3e None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:31.656708 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f2168-3a2a-4c84-a805-958e3b02cb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:31.657214 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f2168-3a2a-4c84-a805-958e3b02cb3e None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:31.658576 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f2168-3a2a-4c84-a805-958e3b02cb3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:31.661489 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f2168-3a2a-4c84-a805-958e3b02cb3e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:31.679556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:31.704433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 241/471] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:16:31 2022] POST /identity/v3/users => generated 248 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:16:31.706210 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f2168-3a2a-4c84-a805-958e3b02cb3e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:31.783253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48951c7-edfa-4cdf-810f-94e20dbb4b62 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:31.868785 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a283c7-d0ec-4ef0-b3ac-493efe401c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:31.869666 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a283c7-d0ec-4ef0-b3ac-493efe401c6a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:31.869963 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a283c7-d0ec-4ef0-b3ac-493efe401c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:31.870246 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a283c7-d0ec-4ef0-b3ac-493efe401c6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:31.904990 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a283c7-d0ec-4ef0-b3ac-493efe401c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:31.906498 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a283c7-d0ec-4ef0-b3ac-493efe401c6a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:31.921123 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f2168-3a2a-4c84-a805-958e3b02cb3e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:31.947244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-01a283c7-d0ec-4ef0-b3ac-493efe401c6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:31.947244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:31.947244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:31.947244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:31.947244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:31.947244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:31.947244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:31.947244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:31.947244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:31.947244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:31.947244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:31.947244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:31.947244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:31.947244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:31.947244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:31.948175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:31.948175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:31.948175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:31.948175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:31.948175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:31.948175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:31.948175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:31.948175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:31.948175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:31.948175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:31.948175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:31.948175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:31.948175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:31.948175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:31.948175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:31.948175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:31.948175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:31.949017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:31.949017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:31.949017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:31.949017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:31.949224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:31.949328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 231/472] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:31.958358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a283c7-d0ec-4ef0-b3ac-493efe401c6a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:31.963553 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c385748-97f6-48a0-85e4-eed5c0f0371f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:31.964450 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c385748-97f6-48a0-85e4-eed5c0f0371f None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:31.964845 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c385748-97f6-48a0-85e4-eed5c0f0371f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:31.965137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c385748-97f6-48a0-85e4-eed5c0f0371f None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:31.966018 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c385748-97f6-48a0-85e4-eed5c0f0371f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:31.967776 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c385748-97f6-48a0-85e4-eed5c0f0371f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:32.016810 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 242/473] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:16:31 2022] POST /identity/v3/users => generated 248 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:16:32.126691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b05ae-c522-41df-a773-ab410cd1af45 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:32.127228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b05ae-c522-41df-a773-ab410cd1af45 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:32.128163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b05ae-c522-41df-a773-ab410cd1af45 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:32.137527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 243/474] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:32.140618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01260c-0725-475e-acb3-2fa9a7a75540 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:32.141132 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01260c-0725-475e-acb3-2fa9a7a75540 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:32.141684 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01260c-0725-475e-acb3-2fa9a7a75540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:32.150669 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a283c7-d0ec-4ef0-b3ac-493efe401c6a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:32.172688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:32.176734 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3f01260c-0725-475e-acb3-2fa9a7a75540 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:32.180702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3f01260c-0725-475e-acb3-2fa9a7a75540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:32.203316 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18c858-d6e2-4ff9-a9c8-f70223dcaffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:32.204107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18c858-d6e2-4ff9-a9c8-f70223dcaffd None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:32.204394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18c858-d6e2-4ff9-a9c8-f70223dcaffd None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:32.204674 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18c858-d6e2-4ff9-a9c8-f70223dcaffd None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:32.205488 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18c858-d6e2-4ff9-a9c8-f70223dcaffd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:32.205803 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18c858-d6e2-4ff9-a9c8-f70223dcaffd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:32.207294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18c858-d6e2-4ff9-a9c8-f70223dcaffd None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:32.221670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 232/475] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:32.226823 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18c858-d6e2-4ff9-a9c8-f70223dcaffd None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:32.321919 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 244/476] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:16:32.350188 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01260c-0725-475e-acb3-2fa9a7a75540 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:32.392979 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18c858-d6e2-4ff9-a9c8-f70223dcaffd None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:32.446314 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5647c-ec1a-4fcc-9659-a265473c7cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:32.447231 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5647c-ec1a-4fcc-9659-a265473c7cad None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:32.447660 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5647c-ec1a-4fcc-9659-a265473c7cad None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:32.448097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5647c-ec1a-4fcc-9659-a265473c7cad None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:32.448964 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5647c-ec1a-4fcc-9659-a265473c7cad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:32.450687 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5647c-ec1a-4fcc-9659-a265473c7cad None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:32.478663 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 233/477] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:16:32 2022] POST /identity/v3/users => generated 249 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:16:32.492842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:32.498930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01260c-0725-475e-acb3-2fa9a7a75540 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:32.545568 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6022cbf-0400-4a39-91db-b169eeabd587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:32.546614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6022cbf-0400-4a39-91db-b169eeabd587 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:32.547096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6022cbf-0400-4a39-91db-b169eeabd587 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:32.547723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6022cbf-0400-4a39-91db-b169eeabd587 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:32.548885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6022cbf-0400-4a39-91db-b169eeabd587 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:32.550762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6022cbf-0400-4a39-91db-b169eeabd587 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:32.563996 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.sql.core [None req-d6022cbf-0400-4a39-91db-b169eeabd587 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 06:16:32.563996 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 06:16:32.563996 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [parameters: {'id': 'd153fb299dcb49289a04f9d081795b9c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 06:16:32.563996 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80280) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 06:16:32.585824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-d6022cbf-0400-4a39-91db-b169eeabd587 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 06:16:32.585824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:32.585824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 06:16:32.585824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 06:16:32.585824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 06:16:32.585824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 06:16:32.585824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 06:16:32.585824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application result = self._query(query) May 19 06:16:32.585824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 06:16:32.585824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application conn.query(q) May 19 06:16:32.585824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 06:16:32.585824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 06:16:32.585824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 06:16:32.585824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application result.read() May 19 06:16:32.585824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return ref.to_dict() May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 06:16:32.586904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application next(self.gen) May 19 06:16:32.587949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 06:16:32.587949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application yield resource May 19 06:16:32.587949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 06:16:32.587949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application next(self.gen) May 19 06:16:32.587949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 06:16:32.587949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.session.rollback() May 19 06:16:32.587949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 06:16:32.587949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.force_reraise() May 19 06:16:32.587949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 06:16:32.587949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise self.value May 19 06:16:32.587949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 06:16:32.587949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 06:16:32.587949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 06:16:32.587949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application session.commit() May 19 06:16:32.587949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 06:16:32.587949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 06:16:32.587949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 06:16:32.588839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._prepare_impl() May 19 06:16:32.588839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 06:16:32.588839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.session.flush() May 19 06:16:32.588839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 06:16:32.588839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._flush(objects) May 19 06:16:32.588839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 06:16:32.588839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 06:16:32.588839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 06:16:32.588839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application compat.raise_( May 19 06:16:32.588839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 06:16:32.588839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception May 19 06:16:32.588839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 06:16:32.588839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application flush_context.execute() May 19 06:16:32.588839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 06:16:32.588839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rec.execute(self) May 19 06:16:32.588839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 06:16:32.588839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 06:16:32.589862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 06:16:32.589862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 06:16:32.589862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 06:16:32.589862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 06:16:32.589862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 06:16:32.589862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 06:16:32.589862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 06:16:32.589862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 06:16:32.589862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 06:16:32.589862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 06:16:32.589862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 06:16:32.589862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 06:16:32.589862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 06:16:32.589862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 06:16:32.589862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 06:16:32.589862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception May 19 06:16:32.591049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 06:16:32.591049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 06:16:32.591049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 06:16:32.591049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 06:16:32.591049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 06:16:32.591049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application result = self._query(query) May 19 06:16:32.591049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 06:16:32.591049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application conn.query(q) May 19 06:16:32.591049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 06:16:32.591049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 06:16:32.591049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 06:16:32.591049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application result.read() May 19 06:16:32.591049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 06:16:32.591049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 06:16:32.591049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 06:16:32.591049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 06:16:32.591911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 06:16:32.591911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 06:16:32.591911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 06:16:32.591911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 06:16:32.591911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 06:16:32.591911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 06:16:32.591911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [parameters: {'id': 'd153fb299dcb49289a04f9d081795b9c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 06:16:32.591911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 06:16:32.591911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:32.591911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:16:32.591911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:32.591911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:32.591911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:32.591911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:32.591911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:32.591911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:32.592720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:32.592720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:32.592720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:32.592720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:32.592720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:32.592720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:32.592720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 06:16:32.592720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 06:16:32.592720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:32.592720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:32.592720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 06:16:32.592720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 06:16:32.592720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 06:16:32.592720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 06:16:32.592720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 06:16:32.592720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:32.595437 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 245/478] 10.222.0.65 () {60 vars in 1131 bytes} [Thu May 19 06:16:32 2022] POST /identity/v3/roles => generated 144 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 06:16:32.601751 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5647c-ec1a-4fcc-9659-a265473c7cad None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:32.675141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:32.797694 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5647c-ec1a-4fcc-9659-a265473c7cad None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:32.858977 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39dd34-ccd9-408d-99ff-12f12e480527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:32.859825 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39dd34-ccd9-408d-99ff-12f12e480527 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:32.860117 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39dd34-ccd9-408d-99ff-12f12e480527 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:32.860401 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39dd34-ccd9-408d-99ff-12f12e480527 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:32.876446 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39dd34-ccd9-408d-99ff-12f12e480527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:32.884961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-ef39dd34-ccd9-408d-99ff-12f12e480527 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:16:32.884961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:32.884961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:32.884961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:32.884961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:32.884961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:32.884961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:32.884961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:32.884961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:32.884961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:32.884961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:32.884961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:32.884961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:32.884961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:32.884961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:32.884961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:32.886052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:32.886052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:32.886052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:32.886052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:32.886052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:32.886052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:32.886052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:32.886052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:32.886052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:32.886052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:32.886052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:32.886052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:32.886052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:32.886052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:32.886052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:32.886052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:32.886847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:32.886847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:32.886847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:32.886847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:32.886847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:32.886847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:32.886847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:32.886847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:32.886847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:32.886847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:32.886847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 06:16:32.886847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:32.886847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 234/479] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:16:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:32.892650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6022cbf-0400-4a39-91db-b169eeabd587 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:33.021606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6022cbf-0400-4a39-91db-b169eeabd587 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:33.063384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b509537-989e-461c-a051-57db41b899b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:33.064394 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b509537-989e-461c-a051-57db41b899b8 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:33.064874 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b509537-989e-461c-a051-57db41b899b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:33.065362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b509537-989e-461c-a051-57db41b899b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:33.066339 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b509537-989e-461c-a051-57db41b899b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:33.066861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b509537-989e-461c-a051-57db41b899b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:33.068455 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b509537-989e-461c-a051-57db41b899b8 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:33.079098 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 246/480] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:33.921817 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50969b-24e3-40f4-8605-31d3698b9edc None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:33.922660 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50969b-24e3-40f4-8605-31d3698b9edc None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:33.922807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50969b-24e3-40f4-8605-31d3698b9edc None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:33.925608 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 235/481] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:33.930932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80f5b1-d668-4ff6-9f9e-8b8586839552 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:33.931648 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80f5b1-d668-4ff6-9f9e-8b8586839552 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:33.931648 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80f5b1-d668-4ff6-9f9e-8b8586839552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:33.958397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ed80f5b1-d668-4ff6-9f9e-8b8586839552 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:33.959617 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ed80f5b1-d668-4ff6-9f9e-8b8586839552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:33.988315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:34.060545 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 247/482] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:34.097794 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50969b-24e3-40f4-8605-31d3698b9edc None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:34.257992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:34.273500 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50969b-24e3-40f4-8605-31d3698b9edc None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:34.376075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0beb523-fa48-4c16-8e87-35c33b9a85ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:34.378060 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0beb523-fa48-4c16-8e87-35c33b9a85ea None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:34.378842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0beb523-fa48-4c16-8e87-35c33b9a85ea None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:34.379599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0beb523-fa48-4c16-8e87-35c33b9a85ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:34.415162 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0beb523-fa48-4c16-8e87-35c33b9a85ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:34.435187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-c0beb523-fa48-4c16-8e87-35c33b9a85ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:34.435187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:34.435187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:34.435187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:34.435187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:34.435187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:34.435187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:34.435187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:34.435187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:34.435187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:34.435187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:34.435187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:34.435187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:34.435187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:34.435187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:34.435187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:34.436978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:34.436978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:34.436978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:34.436978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:34.436978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:34.436978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:34.436978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:34.436978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:34.436978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:34.436978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:34.436978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:34.436978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:34.436978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:34.436978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:34.436978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:34.436978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:34.438599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:34.438599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:34.438599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:34.438599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:34.438599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:34.438599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:34.438599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:34.438599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:34.438599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:34.438599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:34.438599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:34.438599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:34.438599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 236/483] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:34.442731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80f5b1-d668-4ff6-9f9e-8b8586839552 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:34.578471 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80f5b1-d668-4ff6-9f9e-8b8586839552 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:34.625432 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947af244-d4af-4526-9fd0-0a2b30ec23e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:34.626334 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947af244-d4af-4526-9fd0-0a2b30ec23e6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:34.626602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947af244-d4af-4526-9fd0-0a2b30ec23e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:34.626890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947af244-d4af-4526-9fd0-0a2b30ec23e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:34.627674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947af244-d4af-4526-9fd0-0a2b30ec23e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:34.627985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947af244-d4af-4526-9fd0-0a2b30ec23e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:34.629466 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947af244-d4af-4526-9fd0-0a2b30ec23e6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:34.643128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 248/484] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:16:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:34.649035 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0beb523-fa48-4c16-8e87-35c33b9a85ea None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:34.882715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0beb523-fa48-4c16-8e87-35c33b9a85ea None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:34.955950 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468e7a5-52a3-4d57-918c-93c9ce136f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:34.957364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468e7a5-52a3-4d57-918c-93c9ce136f65 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:34.957868 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468e7a5-52a3-4d57-918c-93c9ce136f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:34.958369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468e7a5-52a3-4d57-918c-93c9ce136f65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:34.982299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468e7a5-52a3-4d57-918c-93c9ce136f65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:34.985319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468e7a5-52a3-4d57-918c-93c9ce136f65 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:35.023776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-6468e7a5-52a3-4d57-918c-93c9ce136f65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:35.023776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:35.023776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:35.023776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:35.023776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:35.023776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:35.023776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:35.023776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:35.023776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:35.023776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:35.023776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:35.023776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:35.023776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:35.023776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:35.023776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:35.025847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:35.025847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:35.025847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:35.025847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:35.025847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:35.025847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:35.025847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:35.025847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:35.025847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:35.025847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:35.025847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:35.025847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:35.025847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:35.025847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:35.025847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:35.025847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:35.025847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:35.028016 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:35.028016 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:35.028016 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:35.028016 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:35.028016 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 237/485] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:16:35.038061 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947af244-d4af-4526-9fd0-0a2b30ec23e6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:35.087822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:35.325393 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947af244-d4af-4526-9fd0-0a2b30ec23e6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:35.428239 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cfc61-8af5-47c5-8d87-760710e9800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:35.429470 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cfc61-8af5-47c5-8d87-760710e9800c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:35.429990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cfc61-8af5-47c5-8d87-760710e9800c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:35.430505 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cfc61-8af5-47c5-8d87-760710e9800c None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:35.431932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cfc61-8af5-47c5-8d87-760710e9800c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:35.432479 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cfc61-8af5-47c5-8d87-760710e9800c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:35.435986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cfc61-8af5-47c5-8d87-760710e9800c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:35.452709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bfac1-0c36-4628-82b8-9d5bf537fad9 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:35.453127 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bfac1-0c36-4628-82b8-9d5bf537fad9 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:35.453631 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bfac1-0c36-4628-82b8-9d5bf537fad9 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:35.456218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 238/486] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:16:35.466449 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604faff-8702-4ca5-82d9-334ca57a42a1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:35.466949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604faff-8702-4ca5-82d9-334ca57a42a1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:35.467446 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604faff-8702-4ca5-82d9-334ca57a42a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:35.487061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:35.490910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 249/487] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:35.494305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cfc61-8af5-47c5-8d87-760710e9800c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:35.529205 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9604faff-8702-4ca5-82d9-334ca57a42a1 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:35.531512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9604faff-8702-4ca5-82d9-334ca57a42a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:35.725006 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 239/488] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:35.755606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:35.794869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604faff-8702-4ca5-82d9-334ca57a42a1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:35.805322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cfc61-8af5-47c5-8d87-760710e9800c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:35.894796 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681c381-10df-41b7-b871-e9ef476ded98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:35.895617 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681c381-10df-41b7-b871-e9ef476ded98 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:35.895893 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681c381-10df-41b7-b871-e9ef476ded98 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:35.896167 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681c381-10df-41b7-b871-e9ef476ded98 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:35.914186 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681c381-10df-41b7-b871-e9ef476ded98 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:35.915687 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681c381-10df-41b7-b871-e9ef476ded98 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:35.960451 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604faff-8702-4ca5-82d9-334ca57a42a1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:35.964042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-d681c381-10df-41b7-b871-e9ef476ded98 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 06:16:35.964042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:35.964042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:35.964042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:35.964042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:35.964042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:35.964042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:35.964042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:35.964042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:35.964042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:35.964042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:35.964042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:35.964042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:16:35.964042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:16:35.964042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:16:35.964042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:35.968848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:35.976822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:35.976822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:35.976822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:16:35.976822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:16:35.976822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:16:35.976822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:16:35.976822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:16:35.976822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:16:35.976822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 06:16:35.976822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:35.976822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 250/489] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 06:16:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 478 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:35.983322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681c381-10df-41b7-b871-e9ef476ded98 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:36.007167 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e74122-5ec0-4ad8-8482-a53a79533883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:36.007991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e74122-5ec0-4ad8-8482-a53a79533883 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:36.008250 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e74122-5ec0-4ad8-8482-a53a79533883 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:36.010824 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e74122-5ec0-4ad8-8482-a53a79533883 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:36.017797 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:36.037682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e74122-5ec0-4ad8-8482-a53a79533883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:36.052576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-59e74122-5ec0-4ad8-8482-a53a79533883 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:36.052576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:36.052576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:36.052576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:36.052576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:36.052576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:36.052576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:36.052576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:36.052576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:36.052576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:36.052576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:36.052576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:36.052576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:36.052576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:36.052576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:36.052576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:36.054287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:36.054287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:36.054287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:36.054287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:36.054287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:36.054287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:36.054287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:36.054287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:36.054287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:36.054287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:36.054287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:36.054287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:36.054287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:36.054287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:36.054287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:36.054287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:36.055818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:36.055818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:36.055818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:36.055818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:36.055818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:36.055818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:36.055818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:36.055818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:36.055818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:36.055818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:36.055818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:36.055818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:36.055818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 240/490] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:16:36.063629 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e74122-5ec0-4ad8-8482-a53a79533883 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:36.173195 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681c381-10df-41b7-b871-e9ef476ded98 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:36.206498 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e74122-5ec0-4ad8-8482-a53a79533883 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:36.222592 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7151fc2-1c22-4577-85bb-48d5fd5449cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:36.223234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7151fc2-1c22-4577-85bb-48d5fd5449cd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:36.223519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7151fc2-1c22-4577-85bb-48d5fd5449cd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:36.223807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7151fc2-1c22-4577-85bb-48d5fd5449cd None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:36.224685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7151fc2-1c22-4577-85bb-48d5fd5449cd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:36.224996 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7151fc2-1c22-4577-85bb-48d5fd5449cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:36.226907 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7151fc2-1c22-4577-85bb-48d5fd5449cd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:36.247780 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:36.279248 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 251/491] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 06:16:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:36.282093 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1349b-fce6-463a-8a2e-cac775e663c0 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:36.282605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1349b-fce6-463a-8a2e-cac775e663c0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:36.283106 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1349b-fce6-463a-8a2e-cac775e663c0 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:36.285301 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35de1c-3cca-497d-962f-00175377be0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:36.286031 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35de1c-3cca-497d-962f-00175377be0f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:36.286231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 252/492] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:36.286327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35de1c-3cca-497d-962f-00175377be0f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:36.286656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35de1c-3cca-497d-962f-00175377be0f None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:36.287441 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35de1c-3cca-497d-962f-00175377be0f None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:36.287660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1349b-fce6-463a-8a2e-cac775e663c0 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:36.287824 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35de1c-3cca-497d-962f-00175377be0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:36.289251 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35de1c-3cca-497d-962f-00175377be0f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:36.300813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 241/493] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:16:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:36.302242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad0377-be52-4e2e-8683-a767b4844396 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:36.302539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad0377-be52-4e2e-8683-a767b4844396 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:36.302824 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad0377-be52-4e2e-8683-a767b4844396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:36.335845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-98ad0377-be52-4e2e-8683-a767b4844396 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:36.338460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-98ad0377-be52-4e2e-8683-a767b4844396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:36.430036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:36.451805 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 242/494] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:36.455031 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad0377-be52-4e2e-8683-a767b4844396 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:36.478197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1349b-fce6-463a-8a2e-cac775e663c0 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:36.529416 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eec43a-f6c1-475e-83ef-cbd934e444e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:36.530711 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eec43a-f6c1-475e-83ef-cbd934e444e6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:36.531159 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eec43a-f6c1-475e-83ef-cbd934e444e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:36.531599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eec43a-f6c1-475e-83ef-cbd934e444e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:36.596203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad0377-be52-4e2e-8683-a767b4844396 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:36.635799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af3282-38e6-41f5-ab03-2853bcf8bce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:36.636715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af3282-38e6-41f5-ab03-2853bcf8bce5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:36.637522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af3282-38e6-41f5-ab03-2853bcf8bce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:36.637805 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af3282-38e6-41f5-ab03-2853bcf8bce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:36.645863 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:36.649587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af3282-38e6-41f5-ab03-2853bcf8bce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:36.651127 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af3282-38e6-41f5-ab03-2853bcf8bce5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:36.657407 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eec43a-f6c1-475e-83ef-cbd934e444e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:36.659190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eec43a-f6c1-475e-83ef-cbd934e444e6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:36.674597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-57af3282-38e6-41f5-ab03-2853bcf8bce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:36.674597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:36.674597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:36.674597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:36.674597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:36.674597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:36.674597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:36.674597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:36.674597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:36.674597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:36.674597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:36.674597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:36.674597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:36.674597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:36.674597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:36.675544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:36.675544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:36.675544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:36.675544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:36.675544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:36.675544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:36.675544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:36.675544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:36.675544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:36.675544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:36.675544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:36.675544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:36.675544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:36.675544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:36.675544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:36.675544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:36.675544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:36.676410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:36.676410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:36.676410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:36.676410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:36.681692 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 243/495] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:36.683888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66563236-37a4-43b8-8088-7cb5c9d9dd28 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:36.684437 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66563236-37a4-43b8-8088-7cb5c9d9dd28 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:36.685061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66563236-37a4-43b8-8088-7cb5c9d9dd28 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:36.693203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 244/496] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:36.693203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66563236-37a4-43b8-8088-7cb5c9d9dd28 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:36.770801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:36.791612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 253/497] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:36.793014 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eec43a-f6c1-475e-83ef-cbd934e444e6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:36.874716 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66563236-37a4-43b8-8088-7cb5c9d9dd28 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:36.910590 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:36.915262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13abe9-b8f5-441d-87e3-56a15ea713a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:36.916082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13abe9-b8f5-441d-87e3-56a15ea713a8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:36.916362 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13abe9-b8f5-441d-87e3-56a15ea713a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:36.916652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13abe9-b8f5-441d-87e3-56a15ea713a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:36.920199 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eec43a-f6c1-475e-83ef-cbd934e444e6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:36.930308 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13abe9-b8f5-441d-87e3-56a15ea713a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:36.938652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-ff13abe9-b8f5-441d-87e3-56a15ea713a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:36.938652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:36.938652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:36.938652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:36.938652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:36.938652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:36.938652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:36.938652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:36.938652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:36.938652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:36.938652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:36.938652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:36.938652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:36.938652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:36.938652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:36.938652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:36.948656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:36.948656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:36.948656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:36.948656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:36.948656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:36.948656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:36.948656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:36.948656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:36.948656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:36.948656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:36.948656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:36.948656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:36.948656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:36.948656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:36.948656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:36.948656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:36.950587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:36.950587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:36.950587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:36.950587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:36.950587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:36.950587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:36.950587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:36.950587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:36.950587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:36.950587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:36.950587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:36.950587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:36.950587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 245/498] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:36.950587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05bb69-9e44-40fe-896e-ba563417d98f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:36.952221 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05bb69-9e44-40fe-896e-ba563417d98f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:36.952221 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05bb69-9e44-40fe-896e-ba563417d98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:36.966289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2a05bb69-9e44-40fe-896e-ba563417d98f None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:36.967273 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f117a-79c4-4f1e-ae19-6a7d11596e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:36.967508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2a05bb69-9e44-40fe-896e-ba563417d98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:36.968075 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f117a-79c4-4f1e-ae19-6a7d11596e91 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:36.968448 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f117a-79c4-4f1e-ae19-6a7d11596e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:36.968829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f117a-79c4-4f1e-ae19-6a7d11596e91 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:36.969770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f117a-79c4-4f1e-ae19-6a7d11596e91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:36.970168 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f117a-79c4-4f1e-ae19-6a7d11596e91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:36.971754 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f117a-79c4-4f1e-ae19-6a7d11596e91 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:36.986087 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 254/499] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:36.988297 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75081141-5945-42a0-b406-71cbcdc91eba None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:36.988811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75081141-5945-42a0-b406-71cbcdc91eba None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:36.989334 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75081141-5945-42a0-b406-71cbcdc91eba None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:36.991825 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 255/500] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:36.993211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75081141-5945-42a0-b406-71cbcdc91eba None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:37.057818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 246/501] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:37.059692 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05bb69-9e44-40fe-896e-ba563417d98f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:37.105430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:37.129963 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75081141-5945-42a0-b406-71cbcdc91eba None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:37.172809 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be9c43-9c35-402a-9a07-b5cd7c3688e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:37.174787 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be9c43-9c35-402a-9a07-b5cd7c3688e8 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:37.175559 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be9c43-9c35-402a-9a07-b5cd7c3688e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:37.175827 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be9c43-9c35-402a-9a07-b5cd7c3688e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:37.193188 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be9c43-9c35-402a-9a07-b5cd7c3688e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:37.195277 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be9c43-9c35-402a-9a07-b5cd7c3688e8 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:37.203953 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05bb69-9e44-40fe-896e-ba563417d98f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:37.218436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-72be9c43-9c35-402a-9a07-b5cd7c3688e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:37.218436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:37.218436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:37.218436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:37.218436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:37.218436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:37.218436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:37.218436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:37.218436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:37.218436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:37.218436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:37.218436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:37.218436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:37.218436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:37.218436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:37.220190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:37.220190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:37.220190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:37.220190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:37.220190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:37.220190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:37.220190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:37.220190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:37.220190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:37.220190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:37.220190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:37.220190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:37.220190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:37.220190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:37.220190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:37.220190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:37.220190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:37.222850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:37.222850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:37.222850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:37.222850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:37.222850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 256/502] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:37.222850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be9c43-9c35-402a-9a07-b5cd7c3688e8 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:37.238128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:37.260453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a03f2-d888-4cf0-a525-7d2109fd4013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:37.269351 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a03f2-d888-4cf0-a525-7d2109fd4013 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:37.269351 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a03f2-d888-4cf0-a525-7d2109fd4013 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:37.269351 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a03f2-d888-4cf0-a525-7d2109fd4013 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:37.269351 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a03f2-d888-4cf0-a525-7d2109fd4013 None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:37.269351 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a03f2-d888-4cf0-a525-7d2109fd4013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:37.269351 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a03f2-d888-4cf0-a525-7d2109fd4013 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:37.276400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 247/503] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:16:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:37.279206 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d098fd9-06a9-47e9-a5fe-7f2c6d9c0622 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:37.279970 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d098fd9-06a9-47e9-a5fe-7f2c6d9c0622 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:37.280463 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d098fd9-06a9-47e9-a5fe-7f2c6d9c0622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:37.302586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-8d098fd9-06a9-47e9-a5fe-7f2c6d9c0622 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:37.303819 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8d098fd9-06a9-47e9-a5fe-7f2c6d9c0622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:37.384879 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be9c43-9c35-402a-9a07-b5cd7c3688e8 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:37.393256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 248/504] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:37.394207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d098fd9-06a9-47e9-a5fe-7f2c6d9c0622 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:37.429545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:37.447381 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2297d7e-48f5-4ed9-8308-94b8a93e7478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:37.448205 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2297d7e-48f5-4ed9-8308-94b8a93e7478 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:37.448499 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2297d7e-48f5-4ed9-8308-94b8a93e7478 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:37.448778 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2297d7e-48f5-4ed9-8308-94b8a93e7478 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:37.468972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2297d7e-48f5-4ed9-8308-94b8a93e7478 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:37.474704 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2297d7e-48f5-4ed9-8308-94b8a93e7478 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:37.499368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f2297d7e-48f5-4ed9-8308-94b8a93e7478 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 06:16:37.499368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:37.499368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:37.499368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:37.499368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:37.499368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:37.499368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:37.499368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:37.499368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:37.499368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:37.499368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:37.499368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:37.499368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:16:37.499368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:16:37.499368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:16:37.499368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:37.500982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:37.502669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:37.502669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:37.502669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:16:37.502669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:16:37.502669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:16:37.502669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:16:37.502669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:16:37.502669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:16:37.502669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 06:16:37.502669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:37.502669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 257/505] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:37.502669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2297d7e-48f5-4ed9-8308-94b8a93e7478 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:37.539459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d098fd9-06a9-47e9-a5fe-7f2c6d9c0622 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:37.598946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc9ce1-26d0-4e22-be90-3114cecf7872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:37.599659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc9ce1-26d0-4e22-be90-3114cecf7872 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:37.599946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc9ce1-26d0-4e22-be90-3114cecf7872 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:37.600227 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc9ce1-26d0-4e22-be90-3114cecf7872 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:37.607397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:37.621063 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedc9ce1-26d0-4e22-be90-3114cecf7872 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:37.636350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-aedc9ce1-26d0-4e22-be90-3114cecf7872 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:37.636350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:37.636350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:37.636350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:37.636350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:37.636350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:37.636350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:37.636350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:37.636350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:37.636350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:37.636350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:37.636350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:37.636350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:37.636350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:37.636350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:37.636350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:37.638486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:37.638486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:37.638486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:37.638486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:37.638486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:37.638486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:37.638486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:37.638486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:37.638486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:37.638486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:37.638486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:37.638486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:37.638486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:37.638486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:37.638486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:37.638486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:37.639289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:37.639289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:37.639289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:37.639289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:37.639289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:37.639289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:37.639289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:37.639289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:37.639289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:37.639289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:37.639289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:37.639289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:37.641454 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 249/506] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:37.641700 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc9ce1-26d0-4e22-be90-3114cecf7872 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:37.664201 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2297d7e-48f5-4ed9-8308-94b8a93e7478 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:37.718970 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02cfbd-72c9-4eec-8672-5c846731e0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:37.719766 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02cfbd-72c9-4eec-8672-5c846731e0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:37.720133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02cfbd-72c9-4eec-8672-5c846731e0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:37.720509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02cfbd-72c9-4eec-8672-5c846731e0e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:37.721400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02cfbd-72c9-4eec-8672-5c846731e0e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:37.721799 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02cfbd-72c9-4eec-8672-5c846731e0e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:37.723318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02cfbd-72c9-4eec-8672-5c846731e0e7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:37.738070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 258/507] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:37.739506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02cfbd-72c9-4eec-8672-5c846731e0e7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:37.776413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:37.807859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc9ce1-26d0-4e22-be90-3114cecf7872 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:37.849656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52360bd-5e76-4c5e-b8ee-b2345c714400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:37.850379 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52360bd-5e76-4c5e-b8ee-b2345c714400 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:37.850661 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52360bd-5e76-4c5e-b8ee-b2345c714400 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:37.850939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52360bd-5e76-4c5e-b8ee-b2345c714400 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:37.867938 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52360bd-5e76-4c5e-b8ee-b2345c714400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:37.869639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52360bd-5e76-4c5e-b8ee-b2345c714400 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:37.894144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-b52360bd-5e76-4c5e-b8ee-b2345c714400 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:37.894144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:37.894144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:37.894144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:37.894144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:37.894144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:37.894144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:37.894144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:37.894144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:37.894144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:37.894144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:37.894144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:37.894144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:37.894144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:37.894144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:37.896123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:37.896123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:37.896123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:37.896123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:37.896123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:37.896123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:37.896123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:37.896123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:37.896123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:37.896123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:37.896123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:37.896123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:37.896123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:37.896123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:37.896123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:37.896123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:37.896123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:37.898085 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:37.898085 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:37.898085 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:37.898085 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:37.898085 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 250/508] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:37.898085 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52360bd-5e76-4c5e-b8ee-b2345c714400 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:37.924400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02cfbd-72c9-4eec-8672-5c846731e0e7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:37.959543 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:37.971173 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5954c-a849-44ef-aff7-55e2e5d7a999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:37.972266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5954c-a849-44ef-aff7-55e2e5d7a999 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:37.972558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5954c-a849-44ef-aff7-55e2e5d7a999 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:37.972843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5954c-a849-44ef-aff7-55e2e5d7a999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:37.986217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5954c-a849-44ef-aff7-55e2e5d7a999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:37.988371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5954c-a849-44ef-aff7-55e2e5d7a999 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:38.015321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-03d5954c-a849-44ef-aff7-55e2e5d7a999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:38.015321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:38.015321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:38.015321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:38.015321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:38.015321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:38.015321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:38.015321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:38.015321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:38.015321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:38.015321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:38.015321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:38.015321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:38.015321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:38.015321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:38.023190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:38.023190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:38.023190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:38.023190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:38.023190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:38.023190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:38.023190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:38.023190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:38.023190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:38.023190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:38.023190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:38.023190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:38.023190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:38.023190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:38.023190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:38.023190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:38.023190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:38.024991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:38.024991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:38.024991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:38.024991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:38.024991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 259/509] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:38.024991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5954c-a849-44ef-aff7-55e2e5d7a999 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:38.053235 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52360bd-5e76-4c5e-b8ee-b2345c714400 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:38.097405 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:38.103531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da429df-49c2-4918-b0da-4311a6e9b37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:38.105549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da429df-49c2-4918-b0da-4311a6e9b37c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:38.105549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da429df-49c2-4918-b0da-4311a6e9b37c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:38.105549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da429df-49c2-4918-b0da-4311a6e9b37c None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:38.105995 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da429df-49c2-4918-b0da-4311a6e9b37c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:38.106124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da429df-49c2-4918-b0da-4311a6e9b37c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:38.108374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da429df-49c2-4918-b0da-4311a6e9b37c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:38.151909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 251/510] 10.222.0.65 () {56 vars in 1137 bytes} [Thu May 19 06:16:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:38.153223 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da429df-49c2-4918-b0da-4311a6e9b37c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:38.182288 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5954c-a849-44ef-aff7-55e2e5d7a999 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:38.245727 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c80d6-fbae-4cd5-8c20-d0c7f24cf17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:38.247559 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c80d6-fbae-4cd5-8c20-d0c7f24cf17a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:38.248077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c80d6-fbae-4cd5-8c20-d0c7f24cf17a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:38.248590 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c80d6-fbae-4cd5-8c20-d0c7f24cf17a None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:38.250078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c80d6-fbae-4cd5-8c20-d0c7f24cf17a None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:38.250640 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c80d6-fbae-4cd5-8c20-d0c7f24cf17a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:38.253276 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c80d6-fbae-4cd5-8c20-d0c7f24cf17a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:38.269321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 260/511] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:16:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:38.270734 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c80d6-fbae-4cd5-8c20-d0c7f24cf17a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:38.286513 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da429df-49c2-4918-b0da-4311a6e9b37c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:38.344688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07043811-7ee6-42de-a781-8dbe74ab20d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:38.344970 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:38.346211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07043811-7ee6-42de-a781-8dbe74ab20d3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:38.346823 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07043811-7ee6-42de-a781-8dbe74ab20d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:38.347253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07043811-7ee6-42de-a781-8dbe74ab20d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:38.361047 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07043811-7ee6-42de-a781-8dbe74ab20d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:38.363878 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07043811-7ee6-42de-a781-8dbe74ab20d3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:38.391390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-07043811-7ee6-42de-a781-8dbe74ab20d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:38.391390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:38.391390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:38.391390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:38.391390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:38.391390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:38.391390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:38.391390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:38.391390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:38.391390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:38.391390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:38.391390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:38.391390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:16:38.391390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:16:38.391390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:16:38.392626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:16:38.392626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:38.392626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:38.392626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:38.392626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:38.392626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:38.392626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:38.392626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:38.392626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:38.392626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:38.392626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:38.392626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:38.392626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:38.392626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:38.392626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:38.392626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:38.392626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:38.393857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:16:38.393857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:16:38.393857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:16:38.393857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:16:38.393857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:16:38.393857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:16:38.393857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:38.393857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:38.393857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 252/512] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:38.395457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07043811-7ee6-42de-a781-8dbe74ab20d3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:38.422738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c80d6-fbae-4cd5-8c20-d0c7f24cf17a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:38.466908 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:38.491077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b635ca4-9a9c-4467-a159-87469545f4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:38.492222 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b635ca4-9a9c-4467-a159-87469545f4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:38.492751 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b635ca4-9a9c-4467-a159-87469545f4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:38.493254 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b635ca4-9a9c-4467-a159-87469545f4c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:38.494728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b635ca4-9a9c-4467-a159-87469545f4c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:38.495296 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b635ca4-9a9c-4467-a159-87469545f4c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:38.498081 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b635ca4-9a9c-4467-a159-87469545f4c7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:38.518777 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 261/513] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:38.519764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b635ca4-9a9c-4467-a159-87469545f4c7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:38.542781 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07043811-7ee6-42de-a781-8dbe74ab20d3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:38.581458 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd503cd-d1ac-49a2-9def-081678938d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:38.582216 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd503cd-d1ac-49a2-9def-081678938d37 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:38.582679 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd503cd-d1ac-49a2-9def-081678938d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:38.583037 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd503cd-d1ac-49a2-9def-081678938d37 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:38.583815 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd503cd-d1ac-49a2-9def-081678938d37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:38.584169 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd503cd-d1ac-49a2-9def-081678938d37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:38.585900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd503cd-d1ac-49a2-9def-081678938d37 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:38.601486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 253/514] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:38.602354 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd503cd-d1ac-49a2-9def-081678938d37 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:38.674680 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b635ca4-9a9c-4467-a159-87469545f4c7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:38.676509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:38.735702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b8ebb-0153-4323-b09f-661751cc94f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:38.735947 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd503cd-d1ac-49a2-9def-081678938d37 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:38.736819 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b8ebb-0153-4323-b09f-661751cc94f5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:38.737459 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b8ebb-0153-4323-b09f-661751cc94f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:38.737875 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b8ebb-0153-4323-b09f-661751cc94f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:38.751851 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:38.782570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922a117-cfee-4e27-b64c-a21f25d9ec48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:38.783367 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922a117-cfee-4e27-b64c-a21f25d9ec48 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:38.783647 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922a117-cfee-4e27-b64c-a21f25d9ec48 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:38.783921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922a117-cfee-4e27-b64c-a21f25d9ec48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:38.795493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922a117-cfee-4e27-b64c-a21f25d9ec48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:38.796966 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922a117-cfee-4e27-b64c-a21f25d9ec48 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:38.816211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-9922a117-cfee-4e27-b64c-a21f25d9ec48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:38.816211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:38.816211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:38.816211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:38.816211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:38.816211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:38.816211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:38.816211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:38.816211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:38.816211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:38.816211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:38.816211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:38.816211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:38.816211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:38.816211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:38.817213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:38.817213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:38.817213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:38.817213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:38.817213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:38.817213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:38.817213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:38.817213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:38.817213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:38.817213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:38.817213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:38.817213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:38.817213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:38.817213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:38.817213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:38.817213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:38.817213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:38.818515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:38.818515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:38.818515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:38.818515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:38.818515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 254/515] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:38.818515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922a117-cfee-4e27-b64c-a21f25d9ec48 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:38.849349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b8ebb-0153-4323-b09f-661751cc94f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:38.851193 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b8ebb-0153-4323-b09f-661751cc94f5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:38.951533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922a117-cfee-4e27-b64c-a21f25d9ec48 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:38.981675 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 262/516] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:38.982551 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b8ebb-0153-4323-b09f-661751cc94f5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:38.985644 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:38.991520 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead03851-ec41-47c0-b000-7230f3a452b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:38.992334 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead03851-ec41-47c0-b000-7230f3a452b9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:38.992619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead03851-ec41-47c0-b000-7230f3a452b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:38.993647 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead03851-ec41-47c0-b000-7230f3a452b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:38.993807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead03851-ec41-47c0-b000-7230f3a452b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:38.994066 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead03851-ec41-47c0-b000-7230f3a452b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:38.995537 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead03851-ec41-47c0-b000-7230f3a452b9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:39.016308 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 255/517] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:16:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:39.017156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead03851-ec41-47c0-b000-7230f3a452b9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:39.048660 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:39.142606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b8ebb-0153-4323-b09f-661751cc94f5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:39.181478 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead03851-ec41-47c0-b000-7230f3a452b9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:39.194273 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079de9b-8712-479a-8332-1b9f3d14bf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:39.195244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079de9b-8712-479a-8332-1b9f3d14bf36 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:39.195586 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079de9b-8712-479a-8332-1b9f3d14bf36 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:39.195919 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079de9b-8712-479a-8332-1b9f3d14bf36 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:39.206619 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079de9b-8712-479a-8332-1b9f3d14bf36 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:39.208440 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079de9b-8712-479a-8332-1b9f3d14bf36 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:39.223658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15290f61-d53e-43ec-b1f7-3a83a16aeb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:39.224454 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15290f61-d53e-43ec-b1f7-3a83a16aeb8b None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:39.224824 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15290f61-d53e-43ec-b1f7-3a83a16aeb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:39.225188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15290f61-d53e-43ec-b1f7-3a83a16aeb8b None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:39.226031 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15290f61-d53e-43ec-b1f7-3a83a16aeb8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:39.226735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-7079de9b-8712-479a-8332-1b9f3d14bf36 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 06:16:39.226735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:39.226735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:39.226735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:39.226735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:39.226735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:39.226735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:39.226735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:39.226735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:39.226735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:39.226735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:39.226735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:39.226735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:16:39.226735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:16:39.226735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:16:39.226735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:39.228409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:39.231420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:39.231420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:39.231420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:16:39.231420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:16:39.231420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:16:39.231420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:16:39.231420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:16:39.231420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:16:39.231420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 06:16:39.231420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:39.233127 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15290f61-d53e-43ec-b1f7-3a83a16aeb8b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:39.233272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 263/518] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:39.233272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079de9b-8712-479a-8332-1b9f3d14bf36 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:39.257381 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 256/519] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:16:39 2022] POST /identity/v3/users => generated 247 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:16:39.258955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15290f61-d53e-43ec-b1f7-3a83a16aeb8b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:39.354206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:39.405764 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:39.417191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079de9b-8712-479a-8332-1b9f3d14bf36 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:39.450141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15290f61-d53e-43ec-b1f7-3a83a16aeb8b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:39.470945 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed7b19-c1b8-4fc8-9d5e-8ad5b0faf30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:39.471619 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed7b19-c1b8-4fc8-9d5e-8ad5b0faf30f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:39.472100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed7b19-c1b8-4fc8-9d5e-8ad5b0faf30f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:39.472388 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed7b19-c1b8-4fc8-9d5e-8ad5b0faf30f None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:39.473648 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed7b19-c1b8-4fc8-9d5e-8ad5b0faf30f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:39.474256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed7b19-c1b8-4fc8-9d5e-8ad5b0faf30f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:39.475854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed7b19-c1b8-4fc8-9d5e-8ad5b0faf30f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:39.493745 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 264/520] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:39.495454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed7b19-c1b8-4fc8-9d5e-8ad5b0faf30f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:39.500787 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f180c-2d3a-4b4b-a667-81ca1156c8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:39.502542 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f180c-2d3a-4b4b-a667-81ca1156c8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:39.502542 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f180c-2d3a-4b4b-a667-81ca1156c8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:39.502542 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f180c-2d3a-4b4b-a667-81ca1156c8e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:39.517501 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f180c-2d3a-4b4b-a667-81ca1156c8e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:39.519042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f180c-2d3a-4b4b-a667-81ca1156c8e4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:39.538636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-f15f180c-2d3a-4b4b-a667-81ca1156c8e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:39.538636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:39.538636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:39.538636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:39.538636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:39.538636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:39.538636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:39.538636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:39.538636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:39.538636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:39.538636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:39.538636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:39.538636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:39.538636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:39.538636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:39.540080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:39.540080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:39.540080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:39.540080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:39.540080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:39.540080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:39.540080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:39.540080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:39.540080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:39.540080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:39.540080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:39.540080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:39.540080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:39.540080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:39.540080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:39.540080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:39.540080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:39.541445 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:39.541445 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:39.541445 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:39.541445 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:39.541445 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 257/521] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:39.541445 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f180c-2d3a-4b4b-a667-81ca1156c8e4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:39.621123 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed7b19-c1b8-4fc8-9d5e-8ad5b0faf30f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:39.656228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:39.672116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f180c-2d3a-4b4b-a667-81ca1156c8e4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:39.674676 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b8cef-e5a3-45fb-8330-9d9ec0873790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:39.674676 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b8cef-e5a3-45fb-8330-9d9ec0873790 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:39.674912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b8cef-e5a3-45fb-8330-9d9ec0873790 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:39.676184 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b8cef-e5a3-45fb-8330-9d9ec0873790 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:39.677189 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b8cef-e5a3-45fb-8330-9d9ec0873790 None admin] RBAC: Adding query filter params (role.id=5cc9ca1bc48b4d749d21d7111b14e187, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=2a09f4878eac4d60a9a7dcedba15c05d) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:39.677557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b8cef-e5a3-45fb-8330-9d9ec0873790 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:39.679048 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b8cef-e5a3-45fb-8330-9d9ec0873790 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:39.693591 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 265/522] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:16:39 2022] GET /identity/v3/role_assignments?user.id=2a09f4878eac4d60a9a7dcedba15c05d&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=5cc9ca1bc48b4d749d21d7111b14e187 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:39.693591 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b8cef-e5a3-45fb-8330-9d9ec0873790 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:39.720937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:39.735087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e34ea1-4190-45bf-aecd-fa80f4a01a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:39.735700 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e34ea1-4190-45bf-aecd-fa80f4a01a81 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:39.735992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e34ea1-4190-45bf-aecd-fa80f4a01a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:39.736277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e34ea1-4190-45bf-aecd-fa80f4a01a81 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:39.737575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e34ea1-4190-45bf-aecd-fa80f4a01a81 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:39.737894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e34ea1-4190-45bf-aecd-fa80f4a01a81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:39.740381 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e34ea1-4190-45bf-aecd-fa80f4a01a81 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:39.769609 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 258/523] 10.222.0.65 () {56 vars in 1137 bytes} [Thu May 19 06:16:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:39.770978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e34ea1-4190-45bf-aecd-fa80f4a01a81 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:39.853965 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b8cef-e5a3-45fb-8330-9d9ec0873790 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:39.896716 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bb1dc-d6d5-4da5-b20f-7f039024784a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:39.897567 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bb1dc-d6d5-4da5-b20f-7f039024784a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:39.897851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bb1dc-d6d5-4da5-b20f-7f039024784a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:39.898138 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bb1dc-d6d5-4da5-b20f-7f039024784a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:39.900784 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e34ea1-4190-45bf-aecd-fa80f4a01a81 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:39.907816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bb1dc-d6d5-4da5-b20f-7f039024784a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:39.909337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bb1dc-d6d5-4da5-b20f-7f039024784a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:39.925724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-1d8bb1dc-d6d5-4da5-b20f-7f039024784a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 06:16:39.925724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:39.925724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:39.925724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:39.925724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:39.925724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:39.925724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:39.925724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:39.925724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:39.925724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:39.925724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:39.925724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:39.925724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:16:39.925724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:16:39.925724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:16:39.925724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:39.927476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:39.929126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:39.929126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:39.929126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:16:39.929126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:16:39.929126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:16:39.929126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:16:39.929126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:16:39.929126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:16:39.929126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 06:16:39.929126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:39.929126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 266/524] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:39.929126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bb1dc-d6d5-4da5-b20f-7f039024784a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:39.955543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afc803-518e-4660-a1ed-997b312ea620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:39.956358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afc803-518e-4660-a1ed-997b312ea620 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:39.957019 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afc803-518e-4660-a1ed-997b312ea620 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:39.957327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afc803-518e-4660-a1ed-997b312ea620 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:39.958121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afc803-518e-4660-a1ed-997b312ea620 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:39.958549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afc803-518e-4660-a1ed-997b312ea620 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:39.960429 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afc803-518e-4660-a1ed-997b312ea620 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:39.974369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 259/525] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:39.975246 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afc803-518e-4660-a1ed-997b312ea620 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:39.998313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:40.052471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:40.110450 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bb1dc-d6d5-4da5-b20f-7f039024784a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:40.133008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afc803-518e-4660-a1ed-997b312ea620 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:40.162707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc997d53-ad6d-471a-9932-24f159393a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:40.163500 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc997d53-ad6d-471a-9932-24f159393a50 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:40.163863 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc997d53-ad6d-471a-9932-24f159393a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:40.164219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc997d53-ad6d-471a-9932-24f159393a50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:40.196353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04ba85-db68-4150-a208-2d83c965338b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:40.198442 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04ba85-db68-4150-a208-2d83c965338b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:40.199202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04ba85-db68-4150-a208-2d83c965338b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:40.200848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04ba85-db68-4150-a208-2d83c965338b None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:40.202870 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04ba85-db68-4150-a208-2d83c965338b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:40.203811 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04ba85-db68-4150-a208-2d83c965338b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:40.206833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04ba85-db68-4150-a208-2d83c965338b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:40.250132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 260/526] 10.222.0.65 () {56 vars in 1139 bytes} [Thu May 19 06:16:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:40.252203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04ba85-db68-4150-a208-2d83c965338b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:40.253740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc997d53-ad6d-471a-9932-24f159393a50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:40.255557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc997d53-ad6d-471a-9932-24f159393a50 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:40.317132 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:40.413384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 267/527] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:40.415279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc997d53-ad6d-471a-9932-24f159393a50 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:40.422557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04ba85-db68-4150-a208-2d83c965338b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:40.431602 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:40.466196 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0358c-db30-4812-a4ec-d753a343c4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:40.467024 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0358c-db30-4812-a4ec-d753a343c4dc None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:40.467315 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0358c-db30-4812-a4ec-d753a343c4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:40.467581 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0358c-db30-4812-a4ec-d753a343c4dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:40.476832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0358c-db30-4812-a4ec-d753a343c4dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:40.478534 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0358c-db30-4812-a4ec-d753a343c4dc None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:40.495557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-07d0358c-db30-4812-a4ec-d753a343c4dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:40.495557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:40.495557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:40.495557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:40.495557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:40.495557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:40.495557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:40.495557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:40.495557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:40.495557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:40.495557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:40.495557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:40.495557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:16:40.495557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:16:40.495557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:16:40.498457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:16:40.498457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:40.498457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:40.498457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:40.498457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:40.498457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:40.498457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:40.498457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:40.498457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:40.498457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:40.498457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:40.498457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:40.498457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:40.498457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:40.498457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:40.498457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:40.498457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:40.500604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:16:40.500604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:16:40.500604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:16:40.500604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:16:40.500604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:16:40.500604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:16:40.500604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:40.500604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:40.500604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 261/528] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:40.500604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0358c-db30-4812-a4ec-d753a343c4dc None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:40.625934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc997d53-ad6d-471a-9932-24f159393a50 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:40.630039 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0358c-db30-4812-a4ec-d753a343c4dc None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:40.672704 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac1c49-19b3-4f55-a471-20ed964b879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:40.673248 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8811b3-cce1-4ba0-ac72-b5fa7d803442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:40.673487 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac1c49-19b3-4f55-a471-20ed964b879c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:40.673718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac1c49-19b3-4f55-a471-20ed964b879c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:40.673960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac1c49-19b3-4f55-a471-20ed964b879c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:40.674060 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8811b3-cce1-4ba0-ac72-b5fa7d803442 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:40.674347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8811b3-cce1-4ba0-ac72-b5fa7d803442 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:40.674742 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8811b3-cce1-4ba0-ac72-b5fa7d803442 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:40.690311 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8811b3-cce1-4ba0-ac72-b5fa7d803442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:40.692608 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8811b3-cce1-4ba0-ac72-b5fa7d803442 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:40.713694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:40.717859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-7a8811b3-cce1-4ba0-ac72-b5fa7d803442 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:40.717859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:40.717859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:40.717859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:40.717859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:40.717859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:40.717859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:40.717859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:40.717859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:40.717859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:40.717859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:40.717859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:40.717859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:40.717859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:40.717859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:40.719294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:40.719294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:40.719294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:40.719294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:40.719294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:40.719294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:40.719294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:40.719294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:40.719294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:40.719294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:40.719294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:40.719294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:40.719294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:40.719294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:40.719294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:40.719294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:40.719294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:40.720887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:40.720887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:40.720887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:40.720887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:40.723821 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 262/529] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:40.724825 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8811b3-cce1-4ba0-ac72-b5fa7d803442 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:40.754911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac1c49-19b3-4f55-a471-20ed964b879c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:40.755216 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:40.757182 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac1c49-19b3-4f55-a471-20ed964b879c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:40.877766 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8811b3-cce1-4ba0-ac72-b5fa7d803442 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:40.913653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 268/530] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:40.914531 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac1c49-19b3-4f55-a471-20ed964b879c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:40.943868 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37e68d-2907-4983-8500-c9f117f656cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:40.945215 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37e68d-2907-4983-8500-c9f117f656cc None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:40.945704 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37e68d-2907-4983-8500-c9f117f656cc None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:40.946199 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37e68d-2907-4983-8500-c9f117f656cc None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:40.947701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37e68d-2907-4983-8500-c9f117f656cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:40.948051 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37e68d-2907-4983-8500-c9f117f656cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:40.951659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37e68d-2907-4983-8500-c9f117f656cc None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:40.975941 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 263/531] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:16:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:40.976675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37e68d-2907-4983-8500-c9f117f656cc None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:40.986842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:41.063943 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac1c49-19b3-4f55-a471-20ed964b879c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:41.112422 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c7949-c43f-401b-8917-256b917b3ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:41.114205 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c7949-c43f-401b-8917-256b917b3ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:41.115185 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c7949-c43f-401b-8917-256b917b3ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:41.115721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c7949-c43f-401b-8917-256b917b3ba5 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:41.116640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:41.116954 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c7949-c43f-401b-8917-256b917b3ba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:41.117876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c7949-c43f-401b-8917-256b917b3ba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:41.119955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c7949-c43f-401b-8917-256b917b3ba5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:41.136197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 269/532] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:41.138451 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c7949-c43f-401b-8917-256b917b3ba5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:41.151862 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37e68d-2907-4983-8500-c9f117f656cc None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:41.236043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ea8eb-fae4-4761-9d6f-cf48239f39ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:41.240255 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ea8eb-fae4-4761-9d6f-cf48239f39ad None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:41.241176 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ea8eb-fae4-4761-9d6f-cf48239f39ad None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:41.241990 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ea8eb-fae4-4761-9d6f-cf48239f39ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:41.271734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ea8eb-fae4-4761-9d6f-cf48239f39ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:41.274554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ea8eb-fae4-4761-9d6f-cf48239f39ad None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:41.286439 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c7949-c43f-401b-8917-256b917b3ba5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:41.300205 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:41.310139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-c79ea8eb-fae4-4761-9d6f-cf48239f39ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:41.310139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:41.310139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:41.310139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:41.310139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:41.310139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:41.310139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:41.310139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:41.310139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:41.310139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:41.310139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:41.310139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:41.310139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:41.310139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:41.310139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:41.311092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:41.311092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:41.311092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:41.311092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:41.311092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:41.311092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:41.311092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:41.311092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:41.311092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:41.311092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:41.311092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:41.311092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:41.311092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:41.311092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:41.311092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:41.311092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:41.311092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:41.311953 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:41.311953 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:41.311953 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:41.311953 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:41.320431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 264/533] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:41.320431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ea8eb-fae4-4761-9d6f-cf48239f39ad None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:41.361961 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf1391-7624-4f30-8414-7a57c97dbaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:41.362724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf1391-7624-4f30-8414-7a57c97dbaf2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:41.363156 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf1391-7624-4f30-8414-7a57c97dbaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:41.363587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf1391-7624-4f30-8414-7a57c97dbaf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:41.364758 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf1391-7624-4f30-8414-7a57c97dbaf2 None admin] RBAC: Adding query filter params (role.id=5cc9ca1bc48b4d749d21d7111b14e187, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=a812c3b9b20f44f8814051381a8005b8) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:41.365317 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf1391-7624-4f30-8414-7a57c97dbaf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:41.367412 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf1391-7624-4f30-8414-7a57c97dbaf2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:41.375611 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 270/534] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:16:41 2022] GET /identity/v3/role_assignments?user.id=a812c3b9b20f44f8814051381a8005b8&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=5cc9ca1bc48b4d749d21d7111b14e187 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:41.377015 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf1391-7624-4f30-8414-7a57c97dbaf2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:41.470128 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ea8eb-fae4-4761-9d6f-cf48239f39ad None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:41.574358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:41.591339 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0ddfb-b28f-4f81-947e-aefe7ed65abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:41.592465 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0ddfb-b28f-4f81-947e-aefe7ed65abf None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:41.592766 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0ddfb-b28f-4f81-947e-aefe7ed65abf None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:41.593054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0ddfb-b28f-4f81-947e-aefe7ed65abf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:41.604699 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf1391-7624-4f30-8414-7a57c97dbaf2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:41.631725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0ddfb-b28f-4f81-947e-aefe7ed65abf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:41.636133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0ddfb-b28f-4f81-947e-aefe7ed65abf None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:41.665917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-1ad0ddfb-b28f-4f81-947e-aefe7ed65abf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:41.665917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:41.665917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:41.665917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:41.665917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:41.665917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:41.665917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:41.665917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:41.665917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:41.665917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:41.665917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:41.665917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:41.665917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:16:41.665917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:16:41.665917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:16:41.667738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:16:41.667738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:41.667738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:41.667738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:41.667738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:41.667738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:41.667738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:41.667738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:41.667738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:41.667738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:41.667738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:41.667738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:41.667738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:41.667738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:41.667738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:41.667738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:41.667738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:41.669641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:16:41.669641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:16:41.669641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:16:41.669641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:16:41.669641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:16:41.669641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:16:41.669641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:41.669641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:41.673602 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 265/535] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:41.691610 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0ddfb-b28f-4f81-947e-aefe7ed65abf None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:41.701898 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f73aac-d54a-4c68-b70c-bfddefd40b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:41.703036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f73aac-d54a-4c68-b70c-bfddefd40b49 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:41.703535 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f73aac-d54a-4c68-b70c-bfddefd40b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:41.704038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f73aac-d54a-4c68-b70c-bfddefd40b49 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:41.706009 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f73aac-d54a-4c68-b70c-bfddefd40b49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:41.706331 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f73aac-d54a-4c68-b70c-bfddefd40b49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:41.708177 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f73aac-d54a-4c68-b70c-bfddefd40b49 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:41.727463 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 271/536] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:41.736437 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f73aac-d54a-4c68-b70c-bfddefd40b49 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:41.811079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:41.928674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f73aac-d54a-4c68-b70c-bfddefd40b49 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:41.938956 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0ddfb-b28f-4f81-947e-aefe7ed65abf None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:41.994271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327ef37-da74-4c57-a7c0-9bf1459cc903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:41.995303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327ef37-da74-4c57-a7c0-9bf1459cc903 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:41.995702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327ef37-da74-4c57-a7c0-9bf1459cc903 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:41.996809 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327ef37-da74-4c57-a7c0-9bf1459cc903 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:42.001522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd030d-1efd-455b-864a-0d592247c288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:42.003344 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd030d-1efd-455b-864a-0d592247c288 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:42.003923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd030d-1efd-455b-864a-0d592247c288 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:42.004522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd030d-1efd-455b-864a-0d592247c288 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:42.006206 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd030d-1efd-455b-864a-0d592247c288 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:42.006900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd030d-1efd-455b-864a-0d592247c288 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:42.009313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327ef37-da74-4c57-a7c0-9bf1459cc903 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:42.010440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd030d-1efd-455b-864a-0d592247c288 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:42.011127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327ef37-da74-4c57-a7c0-9bf1459cc903 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:42.031200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-e327ef37-da74-4c57-a7c0-9bf1459cc903 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:42.031200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:42.031200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:42.031200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:42.031200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:42.031200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:42.031200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:42.031200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:42.031200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:42.031200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:42.031200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:42.031200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:42.031200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:16:42.031200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:16:42.031200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:16:42.032104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:16:42.032104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:42.032104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:42.032104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:42.032104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:42.032104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:42.032104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:42.032104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:42.032104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:42.032104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:42.032104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:42.032104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:42.032104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:42.032104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:42.032104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:42.032104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:42.032104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:42.032956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:16:42.032956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:16:42.032956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:16:42.032956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:16:42.032956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:16:42.032956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:16:42.032956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:42.032956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:42.032956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 272/537] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:16:42.034776 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08cc42-3181-4751-82d1-d48e21905af4 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:42.037303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08cc42-3181-4751-82d1-d48e21905af4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:42.037303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08cc42-3181-4751-82d1-d48e21905af4 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:42.037563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 273/538] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:42.040813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 266/539] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:16:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:42.050619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd030d-1efd-455b-864a-0d592247c288 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:42.051146 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08cc42-3181-4751-82d1-d48e21905af4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:42.149685 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:42.217734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd030d-1efd-455b-864a-0d592247c288 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:42.238646 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:42.257204 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08cc42-3181-4751-82d1-d48e21905af4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:42.266284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475e33b-49dc-4a53-98f7-ba9cafc39dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:42.267694 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475e33b-49dc-4a53-98f7-ba9cafc39dfa None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:42.268170 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475e33b-49dc-4a53-98f7-ba9cafc39dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:42.268631 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475e33b-49dc-4a53-98f7-ba9cafc39dfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:42.269857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475e33b-49dc-4a53-98f7-ba9cafc39dfa None admin] RBAC: Adding query filter params (role.id=5cc9ca1bc48b4d749d21d7111b14e187, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=c92d7ec81ddc4da7a33f9dbb756e59b9) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:42.270385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475e33b-49dc-4a53-98f7-ba9cafc39dfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:42.272116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475e33b-49dc-4a53-98f7-ba9cafc39dfa None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:42.280559 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 267/540] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:16:42 2022] GET /identity/v3/role_assignments?user.id=c92d7ec81ddc4da7a33f9dbb756e59b9&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=5cc9ca1bc48b4d749d21d7111b14e187 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:42.282359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c330a6-9544-424c-b49d-0ecccc397abf None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:42.282880 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c330a6-9544-424c-b49d-0ecccc397abf None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:42.283366 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c330a6-9544-424c-b49d-0ecccc397abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:42.303615 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e7c330a6-9544-424c-b49d-0ecccc397abf None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:42.305408 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e7c330a6-9544-424c-b49d-0ecccc397abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:42.351632 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908e2d5-6a0b-41f0-aea5-f5841908e7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:42.353193 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908e2d5-6a0b-41f0-aea5-f5841908e7bd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:42.353830 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908e2d5-6a0b-41f0-aea5-f5841908e7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:42.354562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908e2d5-6a0b-41f0-aea5-f5841908e7bd None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:42.356966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908e2d5-6a0b-41f0-aea5-f5841908e7bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:42.361321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908e2d5-6a0b-41f0-aea5-f5841908e7bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:42.361321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908e2d5-6a0b-41f0-aea5-f5841908e7bd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:42.413503 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 274/541] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:16:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:42.413503 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908e2d5-6a0b-41f0-aea5-f5841908e7bd None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:42.439829 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 268/542] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:42.510682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c330a6-9544-424c-b49d-0ecccc397abf None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:42.571315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908e2d5-6a0b-41f0-aea5-f5841908e7bd None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:42.621732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba89a8-a082-43a6-afb3-d4b93cfd63df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:42.622585 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba89a8-a082-43a6-afb3-d4b93cfd63df None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:42.622893 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba89a8-a082-43a6-afb3-d4b93cfd63df None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:42.623183 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba89a8-a082-43a6-afb3-d4b93cfd63df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:42.624134 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba89a8-a082-43a6-afb3-d4b93cfd63df None admin] RBAC: Adding query filter params (role.id=5cc9ca1bc48b4d749d21d7111b14e187, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=226f4700a516415cbbd272fdfad7af58) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:42.624458 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba89a8-a082-43a6-afb3-d4b93cfd63df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:42.626117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba89a8-a082-43a6-afb3-d4b93cfd63df None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:42.637944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 275/543] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:16:42 2022] GET /identity/v3/role_assignments?user.id=226f4700a516415cbbd272fdfad7af58&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=5cc9ca1bc48b4d749d21d7111b14e187 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:42.641955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:42.720968 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c330a6-9544-424c-b49d-0ecccc397abf None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:42.778725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428ff1e-b380-40f4-a690-66294af314f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:42.780325 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428ff1e-b380-40f4-a690-66294af314f0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:42.780803 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428ff1e-b380-40f4-a690-66294af314f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:42.781315 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428ff1e-b380-40f4-a690-66294af314f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:42.811119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428ff1e-b380-40f4-a690-66294af314f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:42.831564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-0428ff1e-b380-40f4-a690-66294af314f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:42.831564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:42.831564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:42.831564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:42.831564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:42.831564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:42.831564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:42.831564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:42.831564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:42.831564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:42.831564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:42.831564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:42.831564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:42.831564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:42.831564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:42.831564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:42.832543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:42.832543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:42.832543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:42.832543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:42.832543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:42.832543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:42.832543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:42.832543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:42.832543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:42.832543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:42.832543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:42.832543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:42.832543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:42.832543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:42.832543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:42.832543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:42.833369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:42.833369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:42.833369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:42.833369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:42.833369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:42.833369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:42.833369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:42.833369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:42.833369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:42.833369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:42.833369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:42.833369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:42.834498 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 269/544] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:42.841468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba89a8-a082-43a6-afb3-d4b93cfd63df None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:42.861014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaebf40-24c0-4315-9242-5bcc262153c3 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:42.861520 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaebf40-24c0-4315-9242-5bcc262153c3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:42.862182 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaebf40-24c0-4315-9242-5bcc262153c3 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:42.865353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 270/545] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:42.871083 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817f489-eee3-474d-932e-ec23e78a5a1c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:42.872136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817f489-eee3-474d-932e-ec23e78a5a1c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:42.872623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817f489-eee3-474d-932e-ec23e78a5a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:42.894350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a817f489-eee3-474d-932e-ec23e78a5a1c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:42.894984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:42.895572 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a817f489-eee3-474d-932e-ec23e78a5a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:43.038963 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 271/546] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:16:43.089107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817f489-eee3-474d-932e-ec23e78a5a1c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:43.092272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba89a8-a082-43a6-afb3-d4b93cfd63df None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:43.171413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:43.190090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079dcf3-9da8-4001-b3df-98844d45b686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:43.190795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079dcf3-9da8-4001-b3df-98844d45b686 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:43.191094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079dcf3-9da8-4001-b3df-98844d45b686 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:43.191369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079dcf3-9da8-4001-b3df-98844d45b686 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:43.192153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079dcf3-9da8-4001-b3df-98844d45b686 None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:43.192464 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079dcf3-9da8-4001-b3df-98844d45b686 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:43.193919 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079dcf3-9da8-4001-b3df-98844d45b686 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:43.219235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 276/547] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:16:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:43.226288 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079dcf3-9da8-4001-b3df-98844d45b686 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:43.331680 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817f489-eee3-474d-932e-ec23e78a5a1c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:43.407735 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118f371-9b78-42b6-9f41-497aca9463b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:43.409911 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118f371-9b78-42b6-9f41-497aca9463b2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:43.410708 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118f371-9b78-42b6-9f41-497aca9463b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:43.411484 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118f371-9b78-42b6-9f41-497aca9463b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:43.430645 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c118f371-9b78-42b6-9f41-497aca9463b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:43.441123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-c118f371-9b78-42b6-9f41-497aca9463b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:43.441123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:43.441123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:43.441123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:43.441123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:43.441123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:43.441123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:43.441123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:43.441123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:43.441123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:43.441123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:43.441123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:43.441123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:43.441123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:43.441123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:43.441123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:43.442121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:43.442121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:43.442121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:43.442121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:43.442121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:43.442121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:43.442121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:43.442121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:43.442121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:43.442121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:43.442121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:43.442121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:43.442121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:43.442121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:43.442121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:43.442121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:43.442931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:43.442931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:43.442931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:43.442931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:43.442931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:43.442931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:43.442931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:43.442931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:43.442931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:43.442931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:43.442931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:43.442931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:43.447252 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 272/548] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:43.453639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118f371-9b78-42b6-9f41-497aca9463b2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:43.471741 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:43.520728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079dcf3-9da8-4001-b3df-98844d45b686 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:43.571246 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f417ce-2b05-4406-86e2-25f40b758a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:43.572550 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f417ce-2b05-4406-86e2-25f40b758a4e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:43.573426 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f417ce-2b05-4406-86e2-25f40b758a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:43.574840 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f417ce-2b05-4406-86e2-25f40b758a4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:43.595565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f417ce-2b05-4406-86e2-25f40b758a4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:43.597934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f417ce-2b05-4406-86e2-25f40b758a4e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:43.621268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-a3f417ce-2b05-4406-86e2-25f40b758a4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:43.621268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:43.621268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:43.621268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:43.621268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:43.621268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:43.621268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:43.621268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:43.621268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:43.621268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:43.621268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:43.621268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:43.621268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:43.621268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:43.621268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:43.622659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:43.622659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:43.622659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:43.622659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:43.622659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:43.622659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:43.622659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:43.622659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:43.622659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:43.622659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:43.622659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:43.622659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:43.622659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:43.622659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:43.622659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:43.622659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:43.622659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:43.623725 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:43.623725 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:43.623725 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:43.623725 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:43.626506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 277/549] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:43.632234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f417ce-2b05-4406-86e2-25f40b758a4e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:43.673154 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:43.701748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118f371-9b78-42b6-9f41-497aca9463b2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:43.874704 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b576bca-6dcf-4bff-ac99-55773f6dd2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:43.879029 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b576bca-6dcf-4bff-ac99-55773f6dd2ca None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:43.882177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b576bca-6dcf-4bff-ac99-55773f6dd2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:43.883090 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b576bca-6dcf-4bff-ac99-55773f6dd2ca None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:43.884817 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b576bca-6dcf-4bff-ac99-55773f6dd2ca None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:43.889156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b576bca-6dcf-4bff-ac99-55773f6dd2ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:43.894600 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b576bca-6dcf-4bff-ac99-55773f6dd2ca None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:43.908937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f417ce-2b05-4406-86e2-25f40b758a4e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:43.917128 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 273/550] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:16:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:43.929899 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b576bca-6dcf-4bff-ac99-55773f6dd2ca None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:43.977551 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:43.997266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1cd6e-4220-447c-91af-185674344df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:43.999042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1cd6e-4220-447c-91af-185674344df4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:43.999961 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1cd6e-4220-447c-91af-185674344df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:44.000823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1cd6e-4220-447c-91af-185674344df4 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:44.002766 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1cd6e-4220-447c-91af-185674344df4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:44.003964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1cd6e-4220-447c-91af-185674344df4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:44.007043 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1cd6e-4220-447c-91af-185674344df4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:44.035227 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 278/551] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:44.038191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f084192-44fd-420c-95a7-eb576a6ccb7c None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:44.039111 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f084192-44fd-420c-95a7-eb576a6ccb7c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:44.040028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f084192-44fd-420c-95a7-eb576a6ccb7c None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:44.044230 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 279/552] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:44.046432 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f084192-44fd-420c-95a7-eb576a6ccb7c None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:44.132444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b576bca-6dcf-4bff-ac99-55773f6dd2ca None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:44.200821 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84809e1b-0f5b-4fec-8afc-3b82d4ce397a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:44.202184 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84809e1b-0f5b-4fec-8afc-3b82d4ce397a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:44.202667 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84809e1b-0f5b-4fec-8afc-3b82d4ce397a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:44.203171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84809e1b-0f5b-4fec-8afc-3b82d4ce397a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:44.223791 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84809e1b-0f5b-4fec-8afc-3b82d4ce397a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:44.226137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84809e1b-0f5b-4fec-8afc-3b82d4ce397a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:44.263625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-84809e1b-0f5b-4fec-8afc-3b82d4ce397a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:44.263625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:44.263625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:44.263625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:44.263625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:44.263625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:44.263625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:44.263625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:44.263625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:44.263625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:44.263625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:44.263625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:44.263625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:44.263625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:44.263625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:44.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:44.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:44.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:44.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:44.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:44.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:44.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:44.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:44.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:44.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:44.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:44.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:44.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:44.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:44.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:44.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:44.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:44.267097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:44.267097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:44.267097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:44.267097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:44.267097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 274/553] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:16:44.268589 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f97031-a79c-4de2-841a-e12784a2ce2a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:44.269198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f97031-a79c-4de2-841a-e12784a2ce2a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:44.269587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f97031-a79c-4de2-841a-e12784a2ce2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:44.286318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f084192-44fd-420c-95a7-eb576a6ccb7c None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:44.299255 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a6f97031-a79c-4de2-841a-e12784a2ce2a None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:44.301354 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a6f97031-a79c-4de2-841a-e12784a2ce2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:44.305658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:44.361796 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5791b-48b2-4bed-b137-ab1e59db0edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:44.363700 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5791b-48b2-4bed-b137-ab1e59db0edd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:44.364656 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5791b-48b2-4bed-b137-ab1e59db0edd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:44.365495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5791b-48b2-4bed-b137-ab1e59db0edd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:44.382440 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5791b-48b2-4bed-b137-ab1e59db0edd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:44.386094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5791b-48b2-4bed-b137-ab1e59db0edd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:44.419589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-a2a5791b-48b2-4bed-b137-ab1e59db0edd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 06:16:44.419589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:44.419589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:44.419589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:44.419589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:44.419589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:44.419589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:44.419589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:44.419589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:44.419589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:44.419589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:44.419589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:44.419589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:16:44.419589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:16:44.419589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:16:44.419589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:44.421275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:44.423125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:44.423125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:44.423125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:16:44.423125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:16:44.423125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:16:44.423125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:16:44.423125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:16:44.423125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:16:44.423125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 06:16:44.423125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:44.428276 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 280/554] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:16:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:44.430171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5791b-48b2-4bed-b137-ab1e59db0edd None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:44.461633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 275/555] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:44.463281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f97031-a79c-4de2-841a-e12784a2ce2a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:44.535475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:44.657030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5791b-48b2-4bed-b137-ab1e59db0edd None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:44.711562 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f97031-a79c-4de2-841a-e12784a2ce2a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:44.735691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6e377-5b65-4fe0-822d-0b175a2487b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:44.737319 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6e377-5b65-4fe0-822d-0b175a2487b6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:44.738116 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6e377-5b65-4fe0-822d-0b175a2487b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:44.738911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6e377-5b65-4fe0-822d-0b175a2487b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:44.740823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6e377-5b65-4fe0-822d-0b175a2487b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:44.741834 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6e377-5b65-4fe0-822d-0b175a2487b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:44.745178 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6e377-5b65-4fe0-822d-0b175a2487b6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:44.770754 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 281/556] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:44.772784 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6e377-5b65-4fe0-822d-0b175a2487b6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:44.786684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc94b9-2fc4-40e1-b471-c0fce05a067f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:44.787906 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc94b9-2fc4-40e1-b471-c0fce05a067f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:44.788407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc94b9-2fc4-40e1-b471-c0fce05a067f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:44.788911 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc94b9-2fc4-40e1-b471-c0fce05a067f None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:44.790635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc94b9-2fc4-40e1-b471-c0fce05a067f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:44.791196 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc94b9-2fc4-40e1-b471-c0fce05a067f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:44.795488 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc94b9-2fc4-40e1-b471-c0fce05a067f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:44.815100 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:44.843958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 276/557] 10.222.0.65 () {56 vars in 1135 bytes} [Thu May 19 06:16:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:44.846004 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc94b9-2fc4-40e1-b471-c0fce05a067f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:45.013015 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6e377-5b65-4fe0-822d-0b175a2487b6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:45.032066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:45.076732 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc94b9-2fc4-40e1-b471-c0fce05a067f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:45.091223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a001b0b-3922-4aa1-b6b6-0e2c1cac579f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:45.092523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a001b0b-3922-4aa1-b6b6-0e2c1cac579f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:45.093035 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a001b0b-3922-4aa1-b6b6-0e2c1cac579f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:45.093576 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a001b0b-3922-4aa1-b6b6-0e2c1cac579f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:45.118582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a001b0b-3922-4aa1-b6b6-0e2c1cac579f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:45.139326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-7a001b0b-3922-4aa1-b6b6-0e2c1cac579f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:45.139326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:45.139326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:45.139326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:45.139326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:45.139326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:45.139326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:45.139326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:45.139326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:45.139326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:45.139326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:45.139326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:45.139326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:45.139326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:45.139326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:45.139326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:45.141922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:45.141922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:45.141922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:45.141922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:45.141922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:45.141922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:45.141922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:45.141922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:45.141922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:45.141922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:45.141922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:45.141922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:45.141922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:45.141922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:45.141922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:45.141922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:45.143608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:45.143608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:45.143608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:45.143608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:45.143608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:45.143608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:45.143608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:45.143608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:45.143608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:45.143608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:45.143608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:45.143608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:45.148434 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 282/558] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:45.150192 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a001b0b-3922-4aa1-b6b6-0e2c1cac579f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:45.162484 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c4ee7-dfec-4204-84e5-27ddeb5a93bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:45.163304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c4ee7-dfec-4204-84e5-27ddeb5a93bc None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:45.163582 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c4ee7-dfec-4204-84e5-27ddeb5a93bc None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:45.163859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c4ee7-dfec-4204-84e5-27ddeb5a93bc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:45.174031 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c4ee7-dfec-4204-84e5-27ddeb5a93bc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:45.175559 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c4ee7-dfec-4204-84e5-27ddeb5a93bc None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:45.196723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-998c4ee7-dfec-4204-84e5-27ddeb5a93bc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 06:16:45.196723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:45.196723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:45.196723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:45.196723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:45.196723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:45.196723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:45.196723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:45.196723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:45.196723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:45.196723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:45.196723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:45.196723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:16:45.196723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:16:45.196723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:16:45.196723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:45.198036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:45.199112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:45.199112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:45.199112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:16:45.199112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:16:45.199112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:16:45.199112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:16:45.199112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:16:45.199112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:16:45.199112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 06:16:45.199112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:45.199112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 277/559] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 06:16:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:45.200253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c4ee7-dfec-4204-84e5-27ddeb5a93bc None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:45.267463 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:45.323121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c4ee7-dfec-4204-84e5-27ddeb5a93bc None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:45.346534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a001b0b-3922-4aa1-b6b6-0e2c1cac579f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:45.364784 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd454899-eb9e-48f2-bdf0-39c58395b71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:45.365515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd454899-eb9e-48f2-bdf0-39c58395b71f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:45.365809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd454899-eb9e-48f2-bdf0-39c58395b71f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:45.366119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd454899-eb9e-48f2-bdf0-39c58395b71f None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:45.366805 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd454899-eb9e-48f2-bdf0-39c58395b71f None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:45.367111 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd454899-eb9e-48f2-bdf0-39c58395b71f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:45.368526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd454899-eb9e-48f2-bdf0-39c58395b71f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:45.379364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 278/560] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:16:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:45.384888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a944bf-d45c-4012-887e-fa59d991a2bf None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:45.385182 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a944bf-d45c-4012-887e-fa59d991a2bf None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:45.385485 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a944bf-d45c-4012-887e-fa59d991a2bf None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:45.387450 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 279/561] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:45.388176 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a944bf-d45c-4012-887e-fa59d991a2bf None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:45.391945 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd7c3c-7a10-46ad-9dd9-a6d029163ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:45.392692 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd7c3c-7a10-46ad-9dd9-a6d029163ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:45.393051 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd7c3c-7a10-46ad-9dd9-a6d029163ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:45.393451 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd7c3c-7a10-46ad-9dd9-a6d029163ee0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:45.433341 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:45.470611 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd7c3c-7a10-46ad-9dd9-a6d029163ee0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:45.472133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd7c3c-7a10-46ad-9dd9-a6d029163ee0 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:45.504371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a944bf-d45c-4012-887e-fa59d991a2bf None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:45.532785 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:45.564273 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7a4b4-d84c-494c-a7cd-bc5b8cbcc864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:45.565436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7a4b4-d84c-494c-a7cd-bc5b8cbcc864 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:45.565949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7a4b4-d84c-494c-a7cd-bc5b8cbcc864 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:45.566439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7a4b4-d84c-494c-a7cd-bc5b8cbcc864 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:45.568053 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd7a4b4-d84c-494c-a7cd-bc5b8cbcc864 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:45.568618 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd7a4b4-d84c-494c-a7cd-bc5b8cbcc864 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:45.571456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd7a4b4-d84c-494c-a7cd-bc5b8cbcc864 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:45.598325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 283/562] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:45.600625 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd7c3c-7a10-46ad-9dd9-a6d029163ee0 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:45.608015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 280/563] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 06:16:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:45.608015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48038755-4e39-42d9-b1c9-aa53b91d98fe None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:45.608015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48038755-4e39-42d9-b1c9-aa53b91d98fe None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:45.608015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48038755-4e39-42d9-b1c9-aa53b91d98fe None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:45.610253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 281/564] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:45.612132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd72d3b-2ee8-463c-a233-73e8aae2cbeb None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:45.612608 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd72d3b-2ee8-463c-a233-73e8aae2cbeb None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:45.613089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd72d3b-2ee8-463c-a233-73e8aae2cbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:45.641755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-6bd72d3b-2ee8-463c-a233-73e8aae2cbeb None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:45.643928 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6bd72d3b-2ee8-463c-a233-73e8aae2cbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:45.681464 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:45.722098 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd7c3c-7a10-46ad-9dd9-a6d029163ee0 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:45.727062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 282/565] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:45.728452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd72d3b-2ee8-463c-a233-73e8aae2cbeb None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:45.783432 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637a943-c078-487a-8c67-649cbae98147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:45.784157 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637a943-c078-487a-8c67-649cbae98147 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:45.784454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637a943-c078-487a-8c67-649cbae98147 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:45.784728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637a943-c078-487a-8c67-649cbae98147 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:45.796284 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9637a943-c078-487a-8c67-649cbae98147 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:45.797814 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9637a943-c078-487a-8c67-649cbae98147 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:45.817602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-9637a943-c078-487a-8c67-649cbae98147 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:45.817602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:45.817602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:45.817602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:45.817602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:45.817602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:45.817602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:45.817602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:45.817602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:45.817602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:45.817602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:45.817602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:45.817602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:45.817602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:45.817602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:45.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:45.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:45.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:45.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:45.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:45.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:45.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:45.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:45.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:45.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:45.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:45.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:45.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:45.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:45.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:45.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:45.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:45.821163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:45.821163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:45.821163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:45.821163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:45.821163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 284/566] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:45.821163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637a943-c078-487a-8c67-649cbae98147 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:45.894485 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd72d3b-2ee8-463c-a233-73e8aae2cbeb None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:45.905109 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:45.942692 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d07a8-ed83-41f4-aa84-0c410ddf5901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:45.943977 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d07a8-ed83-41f4-aa84-0c410ddf5901 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:45.944447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d07a8-ed83-41f4-aa84-0c410ddf5901 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:45.944907 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d07a8-ed83-41f4-aa84-0c410ddf5901 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:45.954191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637a943-c078-487a-8c67-649cbae98147 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:45.959680 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d07a8-ed83-41f4-aa84-0c410ddf5901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:45.961234 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d07a8-ed83-41f4-aa84-0c410ddf5901 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:45.979848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:45.988350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-c98d07a8-ed83-41f4-aa84-0c410ddf5901 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:45.988350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:45.988350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:45.988350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:45.988350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:45.988350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:45.988350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:45.988350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:45.988350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:45.988350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:45.988350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:45.988350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:45.988350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:45.988350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:45.988350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:45.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:45.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:45.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:45.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:45.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:45.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:45.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:45.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:45.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:45.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:45.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:45.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:45.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:45.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:45.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:45.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:45.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:45.991499 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:45.991499 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:45.991499 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:45.991499 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:45.991499 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 283/567] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:45.993151 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b5420-7ea4-4d0c-98b3-320428a89e8e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:45.993875 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b5420-7ea4-4d0c-98b3-320428a89e8e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:45.994368 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b5420-7ea4-4d0c-98b3-320428a89e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:46.002493 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bebbd6-2071-4a51-859a-53ad67a12fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:46.003517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bebbd6-2071-4a51-859a-53ad67a12fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:46.003892 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bebbd6-2071-4a51-859a-53ad67a12fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:46.004356 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bebbd6-2071-4a51-859a-53ad67a12fb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:46.021523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bebbd6-2071-4a51-859a-53ad67a12fb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:46.021963 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-221b5420-7ea4-4d0c-98b3-320428a89e8e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:46.023810 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-221b5420-7ea4-4d0c-98b3-320428a89e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:46.024054 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bebbd6-2071-4a51-859a-53ad67a12fb1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:46.047570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-c6bebbd6-2071-4a51-859a-53ad67a12fb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:46.047570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:46.047570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:46.047570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:46.047570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:46.047570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:46.047570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:46.047570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:46.047570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:46.047570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:46.047570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:46.047570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:46.047570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:46.047570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:46.047570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:46.049163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:46.049163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:46.049163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:46.049163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:46.049163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:46.049163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:46.049163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:46.049163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:46.049163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:46.049163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:46.049163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:46.049163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:46.049163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:46.049163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:46.049163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:46.049163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:46.049163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:46.050926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:46.050926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:46.050926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:46.050926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:46.050926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 285/568] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:46.050926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bebbd6-2071-4a51-859a-53ad67a12fb1 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:46.132449 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 284/569] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:46.133345 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b5420-7ea4-4d0c-98b3-320428a89e8e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:46.175455 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bebbd6-2071-4a51-859a-53ad67a12fb1 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:46.224167 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042a253-2202-4916-933c-e3798249bf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:46.224970 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042a253-2202-4916-933c-e3798249bf34 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:46.225257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042a253-2202-4916-933c-e3798249bf34 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:46.225565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042a253-2202-4916-933c-e3798249bf34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:46.237587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042a253-2202-4916-933c-e3798249bf34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:46.245689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-d042a253-2202-4916-933c-e3798249bf34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:46.245689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:46.245689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:46.245689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:46.245689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:46.245689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:46.245689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:46.245689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:46.245689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:46.245689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:46.245689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:46.245689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:46.245689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:46.245689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:46.245689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:46.245689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:46.248560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:46.248560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:46.248560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:46.248560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:46.248560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:46.248560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:46.248560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:46.248560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:46.248560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:46.248560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:46.248560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:46.248560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:46.248560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:46.248560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:46.248560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:46.248560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:46.250599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:46.250599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:46.250599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:46.250599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:46.250599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:46.250599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:46.250599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:46.250599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:46.250599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:46.250599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:46.250599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:46.250599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:46.250599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 286/570] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:46.250599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042a253-2202-4916-933c-e3798249bf34 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:46.260389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:46.280710 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b5420-7ea4-4d0c-98b3-320428a89e8e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:46.286932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:46.326373 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a03b4c-65bf-4b82-906b-6f828878225f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:46.327173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03b4c-65bf-4b82-906b-6f828878225f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:46.327601 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03b4c-65bf-4b82-906b-6f828878225f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:46.327945 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03b4c-65bf-4b82-906b-6f828878225f None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:46.329612 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a03b4c-65bf-4b82-906b-6f828878225f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:46.330076 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a03b4c-65bf-4b82-906b-6f828878225f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:46.332384 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a03b4c-65bf-4b82-906b-6f828878225f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:46.347974 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 285/571] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:46.348847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a03b4c-65bf-4b82-906b-6f828878225f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:46.374918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042a253-2202-4916-933c-e3798249bf34 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:46.436735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871125fd-6ced-49ee-b67c-ef3de348863b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:46.437983 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871125fd-6ced-49ee-b67c-ef3de348863b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:46.438500 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871125fd-6ced-49ee-b67c-ef3de348863b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:46.438999 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871125fd-6ced-49ee-b67c-ef3de348863b None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:46.440448 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871125fd-6ced-49ee-b67c-ef3de348863b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:46.441005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871125fd-6ced-49ee-b67c-ef3de348863b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:46.443677 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871125fd-6ced-49ee-b67c-ef3de348863b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:46.462603 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 287/572] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:46.463204 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a03b4c-65bf-4b82-906b-6f828878225f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:46.463656 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871125fd-6ced-49ee-b67c-ef3de348863b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:46.501603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7501c1-b8e4-4c0b-b844-ad844fc4b3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:46.502283 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7501c1-b8e4-4c0b-b844-ad844fc4b3ce None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:46.502581 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7501c1-b8e4-4c0b-b844-ad844fc4b3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:46.502860 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7501c1-b8e4-4c0b-b844-ad844fc4b3ce None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:46.503590 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7501c1-b8e4-4c0b-b844-ad844fc4b3ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:46.503902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7501c1-b8e4-4c0b-b844-ad844fc4b3ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:46.505619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7501c1-b8e4-4c0b-b844-ad844fc4b3ce None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:46.518636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 286/573] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:46.519486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7501c1-b8e4-4c0b-b844-ad844fc4b3ce None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:46.541738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:46.581094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:46.593416 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871125fd-6ced-49ee-b67c-ef3de348863b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:46.641396 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1deda8e-d6b8-4b95-a08d-d44407f80f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:46.642232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deda8e-d6b8-4b95-a08d-d44407f80f4f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:46.642524 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deda8e-d6b8-4b95-a08d-d44407f80f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:46.642796 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deda8e-d6b8-4b95-a08d-d44407f80f4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:46.656103 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1deda8e-d6b8-4b95-a08d-d44407f80f4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:46.664036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-a1deda8e-d6b8-4b95-a08d-d44407f80f4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:46.664036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:46.664036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:46.664036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:46.664036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:46.664036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:46.664036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:46.664036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:46.664036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:46.664036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:46.664036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:46.664036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:46.664036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:46.664036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:46.664036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:46.664036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:46.665080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:46.665080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:46.665080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:46.665080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:46.665080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:46.665080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:46.665080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:46.665080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:46.665080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:46.665080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:46.665080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:46.665080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:46.665080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:46.665080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:46.665080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:46.665080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:46.666153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:46.666153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:46.666153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:46.666153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:46.666153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:46.666153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:46.666153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:46.666153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:46.666153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:46.666153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:46.666153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:46.666153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:46.667218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 288/574] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:46.668271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1deda8e-d6b8-4b95-a08d-d44407f80f4f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:46.678410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7501c1-b8e4-4c0b-b844-ad844fc4b3ce None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:46.736817 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1818a5-0f71-49d5-9469-a1c7d133b102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:46.737563 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1818a5-0f71-49d5-9469-a1c7d133b102 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:46.737850 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1818a5-0f71-49d5-9469-a1c7d133b102 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:46.738133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1818a5-0f71-49d5-9469-a1c7d133b102 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:46.738938 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1818a5-0f71-49d5-9469-a1c7d133b102 None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:46.739251 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1818a5-0f71-49d5-9469-a1c7d133b102 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:46.740763 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1818a5-0f71-49d5-9469-a1c7d133b102 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:46.751226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 287/575] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:16:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:46.752134 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1818a5-0f71-49d5-9469-a1c7d133b102 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:46.815308 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1deda8e-d6b8-4b95-a08d-d44407f80f4f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:46.861924 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:46.876903 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279850c-5271-413a-ae1d-c435f815925a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:46.877931 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279850c-5271-413a-ae1d-c435f815925a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:46.878370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279850c-5271-413a-ae1d-c435f815925a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:46.878802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279850c-5271-413a-ae1d-c435f815925a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:46.890032 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279850c-5271-413a-ae1d-c435f815925a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:46.891701 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279850c-5271-413a-ae1d-c435f815925a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:46.911271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-e279850c-5271-413a-ae1d-c435f815925a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 06:16:46.911271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:46.911271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:46.911271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:46.911271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:46.911271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:46.911271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:46.911271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:46.911271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:46.911271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:46.911271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:46.911271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:46.911271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:16:46.911271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:16:46.911271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:16:46.911271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:46.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:46.915068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:46.915068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:46.915068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:16:46.915068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:16:46.915068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:16:46.915068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:16:46.915068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:16:46.915068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:16:46.915068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 06:16:46.915068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:46.915068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 289/576] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:46.915068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279850c-5271-413a-ae1d-c435f815925a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:46.916361 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1818a5-0f71-49d5-9469-a1c7d133b102 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:46.940576 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:46.955123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d8133-10d4-4e5a-81d2-4739ec5e29ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:46.955935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d8133-10d4-4e5a-81d2-4739ec5e29ee None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:46.956214 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d8133-10d4-4e5a-81d2-4739ec5e29ee None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:46.956487 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d8133-10d4-4e5a-81d2-4739ec5e29ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:46.964308 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d8133-10d4-4e5a-81d2-4739ec5e29ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:46.966389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d8133-10d4-4e5a-81d2-4739ec5e29ee None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:46.982143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-ae4d8133-10d4-4e5a-81d2-4739ec5e29ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:46.982143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:46.982143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:46.982143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:46.982143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:46.982143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:46.982143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:46.982143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:46.982143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:46.982143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:46.982143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:46.982143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:46.982143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:16:46.982143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:16:46.982143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:16:46.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:16:46.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:46.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:46.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:46.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:46.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:46.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:46.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:46.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:46.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:46.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:46.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:46.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:46.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:46.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:46.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:46.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:46.985097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:16:46.985097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:16:46.985097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:16:46.985097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:16:46.985097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:16:46.985097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:16:46.985097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:46.985097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:46.985097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 288/577] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:46.985097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d8133-10d4-4e5a-81d2-4739ec5e29ee None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:47.053382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279850c-5271-413a-ae1d-c435f815925a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:47.092727 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91347aab-91af-4d77-b62a-19b9418a18ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:47.093561 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91347aab-91af-4d77-b62a-19b9418a18ea None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:47.093847 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91347aab-91af-4d77-b62a-19b9418a18ea None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:47.094123 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91347aab-91af-4d77-b62a-19b9418a18ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:47.101601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91347aab-91af-4d77-b62a-19b9418a18ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:47.103076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91347aab-91af-4d77-b62a-19b9418a18ea None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:47.120191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-91347aab-91af-4d77-b62a-19b9418a18ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:47.120191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:47.120191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:47.120191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:47.120191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:47.120191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:47.120191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:47.120191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:47.120191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:47.120191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:47.120191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:47.120191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:47.120191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:16:47.120191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:16:47.120191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:16:47.122574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:16:47.122574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:47.122574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:47.122574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:47.122574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:47.122574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:47.122574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:47.122574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:47.122574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:47.122574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:47.122574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:47.122574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:47.122574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:47.122574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:47.122574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:47.122574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:47.122574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:47.124378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:16:47.124378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:16:47.124378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:16:47.124378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:16:47.124378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:16:47.124378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:16:47.124378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:47.124378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:47.124378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 290/578] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:47.124378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91347aab-91af-4d77-b62a-19b9418a18ea None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:47.140248 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d8133-10d4-4e5a-81d2-4739ec5e29ee None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:47.186219 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:47.193692 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7eda5-bedf-42e9-b91c-90f34b812345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:47.194382 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7eda5-bedf-42e9-b91c-90f34b812345 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:47.194679 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7eda5-bedf-42e9-b91c-90f34b812345 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:47.194960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7eda5-bedf-42e9-b91c-90f34b812345 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:47.195725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7eda5-bedf-42e9-b91c-90f34b812345 None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:47.196036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7eda5-bedf-42e9-b91c-90f34b812345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:47.197474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7eda5-bedf-42e9-b91c-90f34b812345 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:47.206865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 289/579] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:16:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:47.207711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7eda5-bedf-42e9-b91c-90f34b812345 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:47.226463 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:47.273693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91347aab-91af-4d77-b62a-19b9418a18ea None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:47.323229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b33ddb-7248-4d6b-a0f5-01ff49058371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:47.324150 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b33ddb-7248-4d6b-a0f5-01ff49058371 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:47.324503 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b33ddb-7248-4d6b-a0f5-01ff49058371 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:47.325004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b33ddb-7248-4d6b-a0f5-01ff49058371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:47.327600 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7eda5-bedf-42e9-b91c-90f34b812345 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:47.342499 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b33ddb-7248-4d6b-a0f5-01ff49058371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:47.345895 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b33ddb-7248-4d6b-a0f5-01ff49058371 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:47.370008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129339b4-b422-4e3c-a4eb-89fc98df7876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:47.370743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129339b4-b422-4e3c-a4eb-89fc98df7876 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:47.371596 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129339b4-b422-4e3c-a4eb-89fc98df7876 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:47.372535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129339b4-b422-4e3c-a4eb-89fc98df7876 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:47.373429 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129339b4-b422-4e3c-a4eb-89fc98df7876 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:47.373747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129339b4-b422-4e3c-a4eb-89fc98df7876 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:47.375236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129339b4-b422-4e3c-a4eb-89fc98df7876 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:47.381441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-d4b33ddb-7248-4d6b-a0f5-01ff49058371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:47.381441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:47.381441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:47.381441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:47.381441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:47.381441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:47.381441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:47.381441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:47.381441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:47.381441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:47.381441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:47.381441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:47.381441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:47.381441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:47.381441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:47.383294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:47.383294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:47.383294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:47.383294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:47.383294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:47.383294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:47.383294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:47.383294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:47.383294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:47.383294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:47.383294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:47.383294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:47.383294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:47.383294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:47.383294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:47.383294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:47.383294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:47.386279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:47.386279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:47.386279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:47.386279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:47.386279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 291/580] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:47.386279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b33ddb-7248-4d6b-a0f5-01ff49058371 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:47.402142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 290/581] 10.222.0.65 () {56 vars in 1137 bytes} [Thu May 19 06:16:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:47.404319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129339b4-b422-4e3c-a4eb-89fc98df7876 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:47.467321 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:47.533135 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b33ddb-7248-4d6b-a0f5-01ff49058371 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:47.542065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129339b4-b422-4e3c-a4eb-89fc98df7876 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:47.563599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:47.579349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2861c-ba08-4b88-afcb-4b52703e96f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:47.580525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2861c-ba08-4b88-afcb-4b52703e96f2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:47.580882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2861c-ba08-4b88-afcb-4b52703e96f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:47.581233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2861c-ba08-4b88-afcb-4b52703e96f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:47.582187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2861c-ba08-4b88-afcb-4b52703e96f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:47.582516 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2861c-ba08-4b88-afcb-4b52703e96f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:47.584229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2861c-ba08-4b88-afcb-4b52703e96f2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:47.587256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54521771-aabf-4214-8833-51effbd06a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:47.588101 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54521771-aabf-4214-8833-51effbd06a88 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:47.588477 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54521771-aabf-4214-8833-51effbd06a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:47.588842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54521771-aabf-4214-8833-51effbd06a88 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:47.589848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54521771-aabf-4214-8833-51effbd06a88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:47.590263 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54521771-aabf-4214-8833-51effbd06a88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:47.591876 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54521771-aabf-4214-8833-51effbd06a88 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:47.600273 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 292/582] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:16:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:47.600273 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2861c-ba08-4b88-afcb-4b52703e96f2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:47.606487 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 291/583] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:16:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:47.607688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54521771-aabf-4214-8833-51effbd06a88 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:47.734575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54521771-aabf-4214-8833-51effbd06a88 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:47.746605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:47.757407 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2861c-ba08-4b88-afcb-4b52703e96f2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:47.774078 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f13667-5e63-4a4e-8192-74a8b1ef08e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:47.774760 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f13667-5e63-4a4e-8192-74a8b1ef08e6 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:47.775047 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f13667-5e63-4a4e-8192-74a8b1ef08e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:47.775327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f13667-5e63-4a4e-8192-74a8b1ef08e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:47.776060 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f13667-5e63-4a4e-8192-74a8b1ef08e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:47.776372 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f13667-5e63-4a4e-8192-74a8b1ef08e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:47.778269 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f13667-5e63-4a4e-8192-74a8b1ef08e6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:47.791506 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 292/584] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:47.792156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f13667-5e63-4a4e-8192-74a8b1ef08e6 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:47.813951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc3b7b-0329-43fb-8d24-30f3a738ed68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:47.814789 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc3b7b-0329-43fb-8d24-30f3a738ed68 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:47.815084 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc3b7b-0329-43fb-8d24-30f3a738ed68 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:47.815364 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc3b7b-0329-43fb-8d24-30f3a738ed68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:47.828194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc3b7b-0329-43fb-8d24-30f3a738ed68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:47.829718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc3b7b-0329-43fb-8d24-30f3a738ed68 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:47.851860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-fbbc3b7b-0329-43fb-8d24-30f3a738ed68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:47.851860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:47.851860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:47.851860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:47.851860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:47.851860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:47.851860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:47.851860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:47.851860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:47.851860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:47.851860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:47.851860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:47.851860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:47.851860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:47.851860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:47.853774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:47.853774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:47.853774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:47.853774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:47.853774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:47.853774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:47.853774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:47.853774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:47.853774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:47.853774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:47.853774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:47.853774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:47.853774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:47.853774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:47.853774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:47.853774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:47.853774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:47.855083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:47.855083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:47.855083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:47.855083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:47.857527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 293/585] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:47.859411 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc3b7b-0329-43fb-8d24-30f3a738ed68 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:47.922582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:47.937062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f13667-5e63-4a4e-8192-74a8b1ef08e6 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:47.974625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fb56b-e72a-4e8b-99c4-6b1382d7cec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:47.975428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fb56b-e72a-4e8b-99c4-6b1382d7cec2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:47.975719 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fb56b-e72a-4e8b-99c4-6b1382d7cec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:47.976004 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fb56b-e72a-4e8b-99c4-6b1382d7cec2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:47.988694 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5fb56b-e72a-4e8b-99c4-6b1382d7cec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:47.990113 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5fb56b-e72a-4e8b-99c4-6b1382d7cec2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:48.010071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-fa5fb56b-e72a-4e8b-99c4-6b1382d7cec2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:48.010071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:48.010071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:48.010071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:48.010071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:48.010071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:48.010071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:48.010071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:48.010071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:48.010071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:48.010071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:48.010071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:48.010071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:48.010071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:48.010071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:48.012457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:48.012457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:48.012457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:48.012457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:48.012457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:48.012457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:48.012457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:48.012457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:48.012457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:48.012457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:48.012457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:48.012457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:48.012457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:48.012457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:48.012457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:48.012457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:48.012457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:48.014448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:48.014448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:48.014448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:48.014448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:48.014448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 293/586] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:48.014448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fb56b-e72a-4e8b-99c4-6b1382d7cec2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:48.030976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc3b7b-0329-43fb-8d24-30f3a738ed68 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:48.040027 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:48.075583 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a9989-efbb-48d6-a1fe-90ee3395e2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:48.076246 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a9989-efbb-48d6-a1fe-90ee3395e2cc None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:48.076534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a9989-efbb-48d6-a1fe-90ee3395e2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:48.076819 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a9989-efbb-48d6-a1fe-90ee3395e2cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:48.077705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a9989-efbb-48d6-a1fe-90ee3395e2cc None admin] RBAC: Adding query filter params (role.id=5cc9ca1bc48b4d749d21d7111b14e187, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=5fc205d053d54506bb6c9227e4a1b044) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:48.078019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a9989-efbb-48d6-a1fe-90ee3395e2cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:48.079491 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a9989-efbb-48d6-a1fe-90ee3395e2cc None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:48.085468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 294/587] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:16:47 2022] GET /identity/v3/role_assignments?user.id=5fc205d053d54506bb6c9227e4a1b044&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=5cc9ca1bc48b4d749d21d7111b14e187 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:48.086299 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a9989-efbb-48d6-a1fe-90ee3395e2cc None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:48.144096 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fb56b-e72a-4e8b-99c4-6b1382d7cec2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:48.206595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8056e-c4af-490a-957d-7714f1d0d7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:48.208047 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8056e-c4af-490a-957d-7714f1d0d7a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:48.208450 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8056e-c4af-490a-957d-7714f1d0d7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:48.208816 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8056e-c4af-490a-957d-7714f1d0d7a6 None admin] PATH_INFO: `/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/2a09f4878eac4d60a9a7dcedba15c05d/roles/5cc9ca1bc48b4d749d21d7111b14e187` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:48.236354 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8056e-c4af-490a-957d-7714f1d0d7a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3d399dc95424b848b736c0d0298d9a9, user_id=2a09f4878eac4d60a9a7dcedba15c05d, role_id=5cc9ca1bc48b4d749d21d7111b14e187)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:48.239059 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8056e-c4af-490a-957d-7714f1d0d7a6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:48.244002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:48.250212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a9989-efbb-48d6-a1fe-90ee3395e2cc None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:48.264997 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 294/588] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:16:48 2022] PUT /identity/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/2a09f4878eac4d60a9a7dcedba15c05d/roles/5cc9ca1bc48b4d749d21d7111b14e187 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:16:48.266537 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8056e-c4af-490a-957d-7714f1d0d7a6 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:48.339000 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079e656-39bb-4707-b14e-e7c16a28bba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:48.340633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079e656-39bb-4707-b14e-e7c16a28bba2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:48.340778 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079e656-39bb-4707-b14e-e7c16a28bba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:48.341190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079e656-39bb-4707-b14e-e7c16a28bba2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:48.355540 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079e656-39bb-4707-b14e-e7c16a28bba2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:48.358461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079e656-39bb-4707-b14e-e7c16a28bba2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:48.386563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-d079e656-39bb-4707-b14e-e7c16a28bba2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 06:16:48.386563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:48.386563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:48.386563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:48.386563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:48.386563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:48.386563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:48.386563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:48.386563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:48.386563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:48.386563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:48.386563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:48.386563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:16:48.386563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:16:48.386563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:16:48.386563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:48.388373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:48.389477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:48.389477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:48.389477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:16:48.389477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:16:48.389477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:16:48.389477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:16:48.389477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:16:48.389477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:16:48.389477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 06:16:48.389477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:48.389477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 295/589] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:48.390741 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079e656-39bb-4707-b14e-e7c16a28bba2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:48.459525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:48.507723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8056e-c4af-490a-957d-7714f1d0d7a6 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:48.582195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d4ae2-8c81-47a4-8350-565a5e0687a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:48.583343 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d4ae2-8c81-47a4-8350-565a5e0687a2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:48.583956 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d4ae2-8c81-47a4-8350-565a5e0687a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:48.584437 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d4ae2-8c81-47a4-8350-565a5e0687a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:48.585964 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d4ae2-8c81-47a4-8350-565a5e0687a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:48.586519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d4ae2-8c81-47a4-8350-565a5e0687a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:48.589195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d4ae2-8c81-47a4-8350-565a5e0687a2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:48.614596 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079e656-39bb-4707-b14e-e7c16a28bba2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:48.620744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 295/590] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:48.623086 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d4ae2-8c81-47a4-8350-565a5e0687a2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:48.685943 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9c9df-f8cb-4f2c-a4f9-2bc9902e8c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:48.687114 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9c9df-f8cb-4f2c-a4f9-2bc9902e8c37 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:48.687614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9c9df-f8cb-4f2c-a4f9-2bc9902e8c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:48.688117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9c9df-f8cb-4f2c-a4f9-2bc9902e8c37 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:48.689744 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9c9df-f8cb-4f2c-a4f9-2bc9902e8c37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:48.690444 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9c9df-f8cb-4f2c-a4f9-2bc9902e8c37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:48.693095 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9c9df-f8cb-4f2c-a4f9-2bc9902e8c37 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:48.717322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 296/591] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:48.718755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9c9df-f8cb-4f2c-a4f9-2bc9902e8c37 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:48.760871 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:48.819982 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d4ae2-8c81-47a4-8350-565a5e0687a2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:48.879138 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:48.892475 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa8d6c-4162-4b68-b18f-e6bbc04f19bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:48.894083 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa8d6c-4162-4b68-b18f-e6bbc04f19bb None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:48.894547 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa8d6c-4162-4b68-b18f-e6bbc04f19bb None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:48.895009 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa8d6c-4162-4b68-b18f-e6bbc04f19bb None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:48.896567 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa8d6c-4162-4b68-b18f-e6bbc04f19bb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:48.897087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa8d6c-4162-4b68-b18f-e6bbc04f19bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:48.900673 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa8d6c-4162-4b68-b18f-e6bbc04f19bb None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:48.920580 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9c9df-f8cb-4f2c-a4f9-2bc9902e8c37 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:48.944531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 296/592] 10.222.0.65 () {56 vars in 1137 bytes} [Thu May 19 06:16:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:48.946110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa8d6c-4162-4b68-b18f-e6bbc04f19bb None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:49.001166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f3b7c-7077-43e9-83a8-a1765213d26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:49.004189 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f3b7c-7077-43e9-83a8-a1765213d26b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:49.004731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f3b7c-7077-43e9-83a8-a1765213d26b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:49.005223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f3b7c-7077-43e9-83a8-a1765213d26b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:49.021165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f3b7c-7077-43e9-83a8-a1765213d26b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:49.024045 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f3b7c-7077-43e9-83a8-a1765213d26b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:49.055318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-a55f3b7c-7077-43e9-83a8-a1765213d26b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 06:16:49.055318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:49.055318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:49.055318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:49.055318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:49.055318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:49.055318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:49.055318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:49.055318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:49.055318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:49.055318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:49.055318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:49.055318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:16:49.055318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:16:49.055318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:16:49.055318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:49.057154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:49.059788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:49.059788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:49.059788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:16:49.059788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:16:49.059788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:16:49.059788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:16:49.059788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:16:49.059788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:16:49.059788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 06:16:49.059788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:49.059788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 297/593] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:49.059788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f3b7c-7077-43e9-83a8-a1765213d26b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:49.098528 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa8d6c-4162-4b68-b18f-e6bbc04f19bb None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:49.144434 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d592838-2578-4f73-979f-e044c1aa35e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:49.145878 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d592838-2578-4f73-979f-e044c1aa35e2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:49.146249 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d592838-2578-4f73-979f-e044c1aa35e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:49.146602 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d592838-2578-4f73-979f-e044c1aa35e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:49.157575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d592838-2578-4f73-979f-e044c1aa35e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:49.159742 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d592838-2578-4f73-979f-e044c1aa35e2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:49.178576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-0d592838-2578-4f73-979f-e044c1aa35e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:49.178576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:49.178576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:49.178576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:49.178576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:49.178576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:49.178576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:49.178576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:49.178576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:49.178576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:49.178576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:49.178576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:49.178576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:16:49.178576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:16:49.178576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:16:49.181124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:16:49.181124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:49.181124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:49.181124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:49.181124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:49.181124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:49.181124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:49.181124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:49.181124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:49.181124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:49.181124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:49.181124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:49.181124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:49.181124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:49.181124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:49.181124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:49.181124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:49.184390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:16:49.184390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:16:49.184390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:16:49.184390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:16:49.184390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:16:49.184390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:16:49.184390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:49.184390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:49.184390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 297/594] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:49.184390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d592838-2578-4f73-979f-e044c1aa35e2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:49.221458 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:49.243681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f3b7c-7077-43e9-83a8-a1765213d26b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:49.267162 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:49.303635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b7d89-183b-4709-8240-21350c20df59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:49.304901 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b7d89-183b-4709-8240-21350c20df59 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:49.305534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b7d89-183b-4709-8240-21350c20df59 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:49.306040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b7d89-183b-4709-8240-21350c20df59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:49.339448 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b7d89-183b-4709-8240-21350c20df59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:49.343583 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b7d89-183b-4709-8240-21350c20df59 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:49.352682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d592838-2578-4f73-979f-e044c1aa35e2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:49.397578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-a33b7d89-183b-4709-8240-21350c20df59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:49.397578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:49.397578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:49.397578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:49.397578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:49.397578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:49.397578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:49.397578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:49.397578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:49.397578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:49.397578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:49.397578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:49.397578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:49.397578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:49.397578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:49.398555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:49.398555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:49.398555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:49.398555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:49.398555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:49.398555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:49.398555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:49.398555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:49.398555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:49.398555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:49.398555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:49.398555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:49.398555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:49.398555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:49.398555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:49.398555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:49.398555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:49.399443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:49.399443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:49.399443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:49.399443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:49.402580 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4177f8-2d2a-4ba8-b3c1-95ebb8c13e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:49.403636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4177f8-2d2a-4ba8-b3c1-95ebb8c13e68 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:49.404194 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4177f8-2d2a-4ba8-b3c1-95ebb8c13e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:49.404928 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4177f8-2d2a-4ba8-b3c1-95ebb8c13e68 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:49.405620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 298/595] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:49.406308 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4177f8-2d2a-4ba8-b3c1-95ebb8c13e68 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:49.406825 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4177f8-2d2a-4ba8-b3c1-95ebb8c13e68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:49.407247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b7d89-183b-4709-8240-21350c20df59 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:49.408733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4177f8-2d2a-4ba8-b3c1-95ebb8c13e68 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:49.432977 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 298/596] 10.222.0.65 () {56 vars in 1139 bytes} [Thu May 19 06:16:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:49.434298 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4177f8-2d2a-4ba8-b3c1-95ebb8c13e68 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:49.575516 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b7d89-183b-4709-8240-21350c20df59 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:49.591702 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4177f8-2d2a-4ba8-b3c1-95ebb8c13e68 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:49.604883 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:49.618124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:49.626525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5a4b9-928b-4d02-b263-e2903220f675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:49.627563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5a4b9-928b-4d02-b263-e2903220f675 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:49.627986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5a4b9-928b-4d02-b263-e2903220f675 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:49.628542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5a4b9-928b-4d02-b263-e2903220f675 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:49.630256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5a4b9-928b-4d02-b263-e2903220f675 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:49.630537 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5a4b9-928b-4d02-b263-e2903220f675 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:49.633373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5a4b9-928b-4d02-b263-e2903220f675 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:49.639425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca9938-677f-4b7a-832a-1d82d11619c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:49.640116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca9938-677f-4b7a-832a-1d82d11619c7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:49.640406 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca9938-677f-4b7a-832a-1d82d11619c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:49.640677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca9938-677f-4b7a-832a-1d82d11619c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:49.641540 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca9938-677f-4b7a-832a-1d82d11619c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:49.641852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca9938-677f-4b7a-832a-1d82d11619c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:49.644185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca9938-677f-4b7a-832a-1d82d11619c7 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:49.660177 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 299/597] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:16:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:49.661254 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5a4b9-928b-4d02-b263-e2903220f675 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:49.661581 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 299/598] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:49.664253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca9938-677f-4b7a-832a-1d82d11619c7 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:49.827138 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5a4b9-928b-4d02-b263-e2903220f675 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:49.874589 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca9938-677f-4b7a-832a-1d82d11619c7 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:49.899926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587482cc-9842-4b38-aecb-4007ffc454ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:49.900768 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587482cc-9842-4b38-aecb-4007ffc454ce None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:49.901039 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587482cc-9842-4b38-aecb-4007ffc454ce None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:49.901773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587482cc-9842-4b38-aecb-4007ffc454ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:49.916542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587482cc-9842-4b38-aecb-4007ffc454ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:49.918753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587482cc-9842-4b38-aecb-4007ffc454ce None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:49.946019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-587482cc-9842-4b38-aecb-4007ffc454ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:49.946019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:49.946019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:49.946019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:49.946019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:49.946019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:49.946019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:49.946019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:49.946019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:49.946019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:49.946019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:49.946019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:49.946019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:49.946019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:49.946019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:49.948078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:49.948078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:49.948078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:49.948078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:49.948078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:49.948078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:49.948078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:49.948078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:49.948078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:49.948078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:49.948078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:49.948078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:49.948078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:49.948078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:49.948078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:49.948078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:49.948078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:49.951345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:49.951345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:49.951345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:49.951345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:49.951863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54808ac-603c-493e-9c32-2e6ea109d781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:49.952033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 300/599] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:49.952182 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54808ac-603c-493e-9c32-2e6ea109d781 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:49.952182 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54808ac-603c-493e-9c32-2e6ea109d781 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:49.952451 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587482cc-9842-4b38-aecb-4007ffc454ce None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:49.952592 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54808ac-603c-493e-9c32-2e6ea109d781 None admin] PATH_INFO: `/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/a812c3b9b20f44f8814051381a8005b8/roles/5cc9ca1bc48b4d749d21d7111b14e187` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:49.986071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54808ac-603c-493e-9c32-2e6ea109d781 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3d399dc95424b848b736c0d0298d9a9, user_id=a812c3b9b20f44f8814051381a8005b8, role_id=5cc9ca1bc48b4d749d21d7111b14e187)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:49.989313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54808ac-603c-493e-9c32-2e6ea109d781 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:50.028976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 300/600] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:16:49 2022] PUT /identity/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/a812c3b9b20f44f8814051381a8005b8/roles/5cc9ca1bc48b4d749d21d7111b14e187 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:16:50.030989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54808ac-603c-493e-9c32-2e6ea109d781 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:50.047315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:50.097006 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:50.141500 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587482cc-9842-4b38-aecb-4007ffc454ce None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:50.190827 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7f762-6c1a-4f1b-9be2-4d768d926ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:50.191677 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7f762-6c1a-4f1b-9be2-4d768d926ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:50.191968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7f762-6c1a-4f1b-9be2-4d768d926ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:50.192255 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7f762-6c1a-4f1b-9be2-4d768d926ec6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:50.203093 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7f762-6c1a-4f1b-9be2-4d768d926ec6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:50.205002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7f762-6c1a-4f1b-9be2-4d768d926ec6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:50.225105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-16a7f762-6c1a-4f1b-9be2-4d768d926ec6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:50.225105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:50.225105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:50.225105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:50.225105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:50.225105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:50.225105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:50.225105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:50.225105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:50.225105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:50.225105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:50.225105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:50.225105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:16:50.225105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:16:50.225105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:16:50.227220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:16:50.227220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:50.227220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:50.227220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:50.227220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:50.227220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:50.227220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:50.227220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:50.227220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:50.227220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:50.227220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:50.227220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:50.227220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:50.227220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:50.227220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:50.227220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:50.227220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:50.229124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:16:50.229124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:16:50.229124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:16:50.229124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:16:50.229124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:16:50.229124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:16:50.229124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:50.229124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:50.229124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 301/601] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:50.236305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7f762-6c1a-4f1b-9be2-4d768d926ec6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:50.383829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7f762-6c1a-4f1b-9be2-4d768d926ec6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:50.390135 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:50.433946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d7909-1180-4b85-b9c7-249a792fe694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:50.434870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d7909-1180-4b85-b9c7-249a792fe694 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:50.435162 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d7909-1180-4b85-b9c7-249a792fe694 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:50.435327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54808ac-603c-493e-9c32-2e6ea109d781 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:50.435492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d7909-1180-4b85-b9c7-249a792fe694 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:50.436247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d7909-1180-4b85-b9c7-249a792fe694 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:50.436562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d7909-1180-4b85-b9c7-249a792fe694 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:50.439336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d7909-1180-4b85-b9c7-249a792fe694 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:50.454393 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 302/602] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:16:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:50.460516 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d7909-1180-4b85-b9c7-249a792fe694 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:50.515100 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75ed90-ce04-4354-ba7b-49fce37c1f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:50.516270 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75ed90-ce04-4354-ba7b-49fce37c1f09 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:50.516780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75ed90-ce04-4354-ba7b-49fce37c1f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:50.517311 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75ed90-ce04-4354-ba7b-49fce37c1f09 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:50.520050 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75ed90-ce04-4354-ba7b-49fce37c1f09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:50.520615 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75ed90-ce04-4354-ba7b-49fce37c1f09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:50.523364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75ed90-ce04-4354-ba7b-49fce37c1f09 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:50.549474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 301/603] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 520 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:50.559532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75ed90-ce04-4354-ba7b-49fce37c1f09 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:50.592512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d7909-1180-4b85-b9c7-249a792fe694 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:50.639218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef6552-bb46-4262-ba4e-59e95e4b97fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:50.640607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef6552-bb46-4262-ba4e-59e95e4b97fd None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:50.640896 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef6552-bb46-4262-ba4e-59e95e4b97fd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:50.641178 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef6552-bb46-4262-ba4e-59e95e4b97fd None admin] PATH_INFO: `/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/c92d7ec81ddc4da7a33f9dbb756e59b9/roles/5cc9ca1bc48b4d749d21d7111b14e187` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:50.660615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef6552-bb46-4262-ba4e-59e95e4b97fd None admin] RBAC: Authorizing `identity:create_grant(project_id=a3d399dc95424b848b736c0d0298d9a9, user_id=c92d7ec81ddc4da7a33f9dbb756e59b9, role_id=5cc9ca1bc48b4d749d21d7111b14e187)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:50.662209 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef6552-bb46-4262-ba4e-59e95e4b97fd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:50.687460 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 303/604] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:16:50 2022] PUT /identity/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/c92d7ec81ddc4da7a33f9dbb756e59b9/roles/5cc9ca1bc48b4d749d21d7111b14e187 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:16:50.753132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:50.785265 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75ed90-ce04-4354-ba7b-49fce37c1f09 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:50.833502 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7003b-8108-463b-8834-a2d9c3d2bb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:50.834421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7003b-8108-463b-8834-a2d9c3d2bb13 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:50.834712 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7003b-8108-463b-8834-a2d9c3d2bb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:50.835054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7003b-8108-463b-8834-a2d9c3d2bb13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:50.845235 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7003b-8108-463b-8834-a2d9c3d2bb13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:50.846894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7003b-8108-463b-8834-a2d9c3d2bb13 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:50.864466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-f8c7003b-8108-463b-8834-a2d9c3d2bb13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:50.864466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:50.864466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:50.864466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:50.864466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:50.864466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:50.864466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:50.864466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:50.864466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:50.864466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:50.864466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:50.864466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:50.864466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:16:50.864466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:16:50.864466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:16:50.866430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:16:50.866430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:50.866430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:50.866430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:50.866430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:50.866430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:50.866430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:50.866430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:50.866430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:50.866430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:50.866430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:50.866430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:50.866430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:50.866430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:50.866430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:50.866430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:50.866430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:50.867539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:16:50.867539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:16:50.867539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:16:50.867539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:16:50.867539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:16:50.867539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:16:50.867539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:50.867539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:50.867539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 302/605] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:50.875502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef6552-bb46-4262-ba4e-59e95e4b97fd None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:50.909448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b00ed-b341-4823-938a-b33b9c361f4e None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:50.909448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b00ed-b341-4823-938a-b33b9c361f4e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:50.909448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b00ed-b341-4823-938a-b33b9c361f4e None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:50.910977 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 303/606] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:50.917111 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa44c80-cf56-4295-8079-2af813eca77e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:50.917742 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa44c80-cf56-4295-8079-2af813eca77e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:50.918187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa44c80-cf56-4295-8079-2af813eca77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:50.929084 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:50.957695 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5fa44c80-cf56-4295-8079-2af813eca77e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:50.959823 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5fa44c80-cf56-4295-8079-2af813eca77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:51.128143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 304/607] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:16:51.145256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef6552-bb46-4262-ba4e-59e95e4b97fd None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:51.149222 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa44c80-cf56-4295-8079-2af813eca77e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:51.248352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d8a48-d2fd-4bb7-bf5a-100eecab6e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:51.248944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d8a48-d2fd-4bb7-bf5a-100eecab6e17 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:51.249231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d8a48-d2fd-4bb7-bf5a-100eecab6e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:51.257735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d8a48-d2fd-4bb7-bf5a-100eecab6e17 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:51.258641 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d8a48-d2fd-4bb7-bf5a-100eecab6e17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:51.258969 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d8a48-d2fd-4bb7-bf5a-100eecab6e17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:51.260463 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d8a48-d2fd-4bb7-bf5a-100eecab6e17 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:51.265601 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:51.289770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 304/608] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:16:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:16:51.300261 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d8a48-d2fd-4bb7-bf5a-100eecab6e17 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:51.304887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa44c80-cf56-4295-8079-2af813eca77e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:51.352825 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380c527-f684-4d59-8e21-797fa2663ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:51.353726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380c527-f684-4d59-8e21-797fa2663ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:51.354010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380c527-f684-4d59-8e21-797fa2663ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:51.354283 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380c527-f684-4d59-8e21-797fa2663ed6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:51.380929 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380c527-f684-4d59-8e21-797fa2663ed6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:51.398171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-d380c527-f684-4d59-8e21-797fa2663ed6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:51.398171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:51.398171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:51.398171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:51.398171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:51.398171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:51.398171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:51.398171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:51.398171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:51.398171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:51.398171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:51.398171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:51.398171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:51.398171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:51.398171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:51.398171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:51.400121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:51.400121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:51.400121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:51.400121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:51.400121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:51.400121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:51.400121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:51.400121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:51.400121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:51.400121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:51.400121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:51.400121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:51.400121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:51.400121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:51.400121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:51.400121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:51.401718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:51.401718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:51.401718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:51.401718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:51.401718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:51.401718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:51.401718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:51.401718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:51.401718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:51.401718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:51.401718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:51.401718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:51.407380 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 305/609] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:16:51.421596 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380c527-f684-4d59-8e21-797fa2663ed6 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:51.580805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:51.671743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d8a48-d2fd-4bb7-bf5a-100eecab6e17 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:51.773102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380c527-f684-4d59-8e21-797fa2663ed6 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:51.781107 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51fcad-ed52-4b42-80f6-18b3725c605c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:51.783623 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51fcad-ed52-4b42-80f6-18b3725c605c None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:51.783623 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51fcad-ed52-4b42-80f6-18b3725c605c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:51.783851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51fcad-ed52-4b42-80f6-18b3725c605c None admin] PATH_INFO: `/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/226f4700a516415cbbd272fdfad7af58/roles/5cc9ca1bc48b4d749d21d7111b14e187` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:51.815104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51fcad-ed52-4b42-80f6-18b3725c605c None admin] RBAC: Authorizing `identity:create_grant(project_id=a3d399dc95424b848b736c0d0298d9a9, user_id=226f4700a516415cbbd272fdfad7af58, role_id=5cc9ca1bc48b4d749d21d7111b14e187)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:51.818098 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51fcad-ed52-4b42-80f6-18b3725c605c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:51.831923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:51.862139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 305/610] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:16:51 2022] PUT /identity/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/226f4700a516415cbbd272fdfad7af58/roles/5cc9ca1bc48b4d749d21d7111b14e187 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:16:51.864730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cabcb7-ea96-4c7b-a989-6296601a5b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:51.865987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cabcb7-ea96-4c7b-a989-6296601a5b0f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:51.866474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cabcb7-ea96-4c7b-a989-6296601a5b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:51.866941 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cabcb7-ea96-4c7b-a989-6296601a5b0f None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:51.868455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cabcb7-ea96-4c7b-a989-6296601a5b0f None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:51.869064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cabcb7-ea96-4c7b-a989-6296601a5b0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:51.871913 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cabcb7-ea96-4c7b-a989-6296601a5b0f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:51.890378 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 306/611] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:16:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 470 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:51.900445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51fcad-ed52-4b42-80f6-18b3725c605c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:52.125910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51fcad-ed52-4b42-80f6-18b3725c605c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:52.172607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:52.209774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78712fed-c91d-4bea-9de0-07dee0310c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:52.211861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78712fed-c91d-4bea-9de0-07dee0310c92 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:52.212739 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78712fed-c91d-4bea-9de0-07dee0310c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:52.213614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78712fed-c91d-4bea-9de0-07dee0310c92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:52.244584 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78712fed-c91d-4bea-9de0-07dee0310c92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:52.248089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78712fed-c91d-4bea-9de0-07dee0310c92 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:52.298861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-78712fed-c91d-4bea-9de0-07dee0310c92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:52.298861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:52.298861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:52.298861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:52.298861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:52.298861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:52.298861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:52.298861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:52.298861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:52.298861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:52.298861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:52.298861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:52.298861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:52.298861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:52.298861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:52.300712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:52.300712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:52.300712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:52.300712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:52.300712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:52.300712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:52.300712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:52.300712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:52.300712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:52.300712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:52.300712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:52.300712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:52.300712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:52.300712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:52.300712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:52.300712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:52.300712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:52.305690 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:52.305690 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:52.305690 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:52.305690 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:52.309362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 306/612] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:52.314702 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cabcb7-ea96-4c7b-a989-6296601a5b0f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:52.422981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9f019-1eb6-4a55-86e9-416b1946c8f6 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:52.423490 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9f019-1eb6-4a55-86e9-416b1946c8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:52.424294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9f019-1eb6-4a55-86e9-416b1946c8f6 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:52.435214 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 307/613] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:16:52.438608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6112e-3887-4c0c-b64f-4a4c58d073db None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:52.439552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6112e-3887-4c0c-b64f-4a4c58d073db None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:52.440378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6112e-3887-4c0c-b64f-4a4c58d073db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:52.475198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-44a6112e-3887-4c0c-b64f-4a4c58d073db None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:52.477763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-44a6112e-3887-4c0c-b64f-4a4c58d073db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:52.530505 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cabcb7-ea96-4c7b-a989-6296601a5b0f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:52.603734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12eeff8-4ce2-466b-90f3-d794d0428aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:52.604429 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12eeff8-4ce2-466b-90f3-d794d0428aae None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:52.604719 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12eeff8-4ce2-466b-90f3-d794d0428aae None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:52.605525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12eeff8-4ce2-466b-90f3-d794d0428aae None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:52.606285 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12eeff8-4ce2-466b-90f3-d794d0428aae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:52.606690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12eeff8-4ce2-466b-90f3-d794d0428aae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:52.608136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12eeff8-4ce2-466b-90f3-d794d0428aae None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:52.631524 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 307/614] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:52.631926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 308/615] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:16:52.639419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12eeff8-4ce2-466b-90f3-d794d0428aae None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:52.701397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6112e-3887-4c0c-b64f-4a4c58d073db None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:52.766598 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:52.880068 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12eeff8-4ce2-466b-90f3-d794d0428aae None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:52.927324 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6112e-3887-4c0c-b64f-4a4c58d073db None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:52.940063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:52.959740 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e5cf2-2f4f-4a07-a740-b0b1c422c42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:52.960696 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e5cf2-2f4f-4a07-a740-b0b1c422c42d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:52.961034 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e5cf2-2f4f-4a07-a740-b0b1c422c42d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:52.961339 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e5cf2-2f4f-4a07-a740-b0b1c422c42d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:52.984532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e5cf2-2f4f-4a07-a740-b0b1c422c42d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:52.988014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e5cf2-2f4f-4a07-a740-b0b1c422c42d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:53.026471 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e3552-bd75-49b3-91a5-98285599595e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:53.027960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e3552-bd75-49b3-91a5-98285599595e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:53.034705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e3552-bd75-49b3-91a5-98285599595e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:53.034705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e3552-bd75-49b3-91a5-98285599595e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:53.040409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-269e5cf2-2f4f-4a07-a740-b0b1c422c42d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 06:16:53.040409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:53.040409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:53.040409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:53.040409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:53.040409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:53.040409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:53.040409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:53.040409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:53.040409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:53.040409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:53.040409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:53.040409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:16:53.040409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:16:53.040409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:16:53.040409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:53.043084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:53.045355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:53.045355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:53.045355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:16:53.045355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:16:53.045355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:16:53.045355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:16:53.045355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:16:53.045355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:16:53.045355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 06:16:53.045355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:53.051120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 308/616] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:16:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:53.062890 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e5cf2-2f4f-4a07-a740-b0b1c422c42d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:53.090634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e3552-bd75-49b3-91a5-98285599595e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:53.111527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-a16e3552-bd75-49b3-91a5-98285599595e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:53.111527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:53.111527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:53.111527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:53.111527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:53.111527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:53.111527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:53.111527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:53.111527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:53.111527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:53.111527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:53.111527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:53.111527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:53.111527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:53.111527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:53.111527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:53.113517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:53.113517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:53.113517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:53.113517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:53.113517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:53.113517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:53.113517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:53.113517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:53.113517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:53.113517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:53.113517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:53.113517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:53.113517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:53.113517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:53.113517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:53.113517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:53.115472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:53.115472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:53.115472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:53.115472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:53.115472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:53.115472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:53.115472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:53.115472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:53.115472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:53.115472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:53.115472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:53.115472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:53.123677 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 309/617] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:53.135063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e3552-bd75-49b3-91a5-98285599595e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:53.285609 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e5cf2-2f4f-4a07-a740-b0b1c422c42d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:53.312379 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:53.379860 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a79b4-bb65-4f0c-9f8b-6024a1cf5ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:53.381750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a79b4-bb65-4f0c-9f8b-6024a1cf5ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:53.382691 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a79b4-bb65-4f0c-9f8b-6024a1cf5ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:53.383463 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a79b4-bb65-4f0c-9f8b-6024a1cf5ab9 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:53.385484 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a79b4-bb65-4f0c-9f8b-6024a1cf5ab9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:53.386913 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a79b4-bb65-4f0c-9f8b-6024a1cf5ab9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:53.390221 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a79b4-bb65-4f0c-9f8b-6024a1cf5ab9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:53.413328 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e3552-bd75-49b3-91a5-98285599595e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:53.434021 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 309/618] 10.222.0.65 () {56 vars in 1135 bytes} [Thu May 19 06:16:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:53.434021 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b83e3-9f9d-4962-b7e0-b22fa7bd1bcb None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:53.434021 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b83e3-9f9d-4962-b7e0-b22fa7bd1bcb None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:53.434021 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b83e3-9f9d-4962-b7e0-b22fa7bd1bcb None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:53.439589 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 310/619] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:53.441647 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b83e3-9f9d-4962-b7e0-b22fa7bd1bcb None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:53.499982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b726e6-b5aa-4a0c-b87a-65b6c9540b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:53.501626 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b726e6-b5aa-4a0c-b87a-65b6c9540b34 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:53.502420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b726e6-b5aa-4a0c-b87a-65b6c9540b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:53.503176 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b726e6-b5aa-4a0c-b87a-65b6c9540b34 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:53.504873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b726e6-b5aa-4a0c-b87a-65b6c9540b34 None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:53.505814 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b726e6-b5aa-4a0c-b87a-65b6c9540b34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:53.508815 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b726e6-b5aa-4a0c-b87a-65b6c9540b34 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:53.527788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 310/620] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:16:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:53.530834 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d4886-e2b1-43a4-920a-6355fc8effbb None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:53.531636 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d4886-e2b1-43a4-920a-6355fc8effbb None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:53.532366 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d4886-e2b1-43a4-920a-6355fc8effbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:53.562842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d74d4886-e2b1-43a4-920a-6355fc8effbb None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:53.565515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d74d4886-e2b1-43a4-920a-6355fc8effbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:53.612835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:53.668535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b83e3-9f9d-4962-b7e0-b22fa7bd1bcb None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:53.695734 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 311/621] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:53.696971 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d4886-e2b1-43a4-920a-6355fc8effbb None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:53.791895 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c56fc-287f-4252-a095-b5d033a0b3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:53.794690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c56fc-287f-4252-a095-b5d033a0b3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:53.795195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c56fc-287f-4252-a095-b5d033a0b3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:53.795680 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c56fc-287f-4252-a095-b5d033a0b3a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:53.811583 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d4886-e2b1-43a4-920a-6355fc8effbb None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:53.821319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c56fc-287f-4252-a095-b5d033a0b3a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:53.824071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c56fc-287f-4252-a095-b5d033a0b3a0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:53.851463 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d6df3-d704-4246-8f4f-b21d1f120ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:53.852687 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d6df3-d704-4246-8f4f-b21d1f120ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:53.853188 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d6df3-d704-4246-8f4f-b21d1f120ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:53.853730 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d6df3-d704-4246-8f4f-b21d1f120ac3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:53.862923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-af1c56fc-287f-4252-a095-b5d033a0b3a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:53.862923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:53.862923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:53.862923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:53.862923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:53.862923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:53.862923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:53.862923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:53.862923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:53.862923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:53.862923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:53.862923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:53.862923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:53.862923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:53.862923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:53.863892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:53.863892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:53.863892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:53.863892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:53.863892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:53.863892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:53.863892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:53.863892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:53.863892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:53.863892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:53.863892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:53.863892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:53.863892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:53.863892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:53.863892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:53.863892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:53.863892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:53.864780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:53.864780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:53.864780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:53.864780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:53.869019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d6df3-d704-4246-8f4f-b21d1f120ac3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:53.869986 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 311/622] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:53.871919 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d6df3-d704-4246-8f4f-b21d1f120ac3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:53.873016 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c56fc-287f-4252-a095-b5d033a0b3a0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:53.892398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-fd1d6df3-d704-4246-8f4f-b21d1f120ac3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:53.892398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:53.892398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:53.892398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:53.892398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:53.892398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:53.892398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:53.892398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:53.892398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:53.892398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:53.892398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:53.892398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:53.892398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:53.892398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:53.892398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:53.893424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:53.893424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:53.893424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:53.893424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:53.893424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:53.893424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:53.893424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:53.893424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:53.893424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:53.893424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:53.893424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:53.893424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:53.893424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:53.893424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:53.893424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:53.893424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:53.893424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:53.894325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:53.894325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:53.894325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:53.894325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:53.896714 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 312/623] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:53.897847 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d6df3-d704-4246-8f4f-b21d1f120ac3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:53.918459 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:53.921723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:54.018941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d6df3-d704-4246-8f4f-b21d1f120ac3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:54.061235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4366d88-ad2b-4561-9b61-499b843c752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:54.062472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4366d88-ad2b-4561-9b61-499b843c752e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:54.062985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4366d88-ad2b-4561-9b61-499b843c752e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:54.063485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4366d88-ad2b-4561-9b61-499b843c752e None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:54.064528 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4366d88-ad2b-4561-9b61-499b843c752e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:54.065054 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4366d88-ad2b-4561-9b61-499b843c752e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:54.066720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4366d88-ad2b-4561-9b61-499b843c752e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:54.081005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 313/624] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:54.082208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4366d88-ad2b-4561-9b61-499b843c752e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:54.099231 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c56fc-287f-4252-a095-b5d033a0b3a0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:54.136923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2d48c-42b2-4c9f-bdf3-8229f9546922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:54.137791 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2d48c-42b2-4c9f-bdf3-8229f9546922 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:54.138096 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2d48c-42b2-4c9f-bdf3-8229f9546922 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:54.138347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2d48c-42b2-4c9f-bdf3-8229f9546922 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:54.150053 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2d48c-42b2-4c9f-bdf3-8229f9546922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:54.158869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-78c2d48c-42b2-4c9f-bdf3-8229f9546922 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:54.158869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:54.158869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:54.158869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:54.158869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:54.158869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:54.158869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:54.158869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:54.158869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:54.158869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:54.158869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:54.158869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:54.158869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:54.158869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:54.158869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:54.158869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:54.159874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:54.159874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:54.159874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:54.159874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:54.159874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:54.159874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:54.159874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:54.159874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:54.159874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:54.159874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:54.159874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:54.159874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:54.159874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:54.159874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:54.159874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:54.159874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:54.160734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:54.160734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:54.160734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:54.160734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:54.160734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:54.160734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:54.160734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:54.160734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:54.160734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:54.160734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:54.160734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:54.160734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:54.165318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 312/625] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:54.166252 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2d48c-42b2-4c9f-bdf3-8229f9546922 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:54.188755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:54.235659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4366d88-ad2b-4561-9b61-499b843c752e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:54.273820 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55dcc2-3778-4e75-9002-6a5c2d8126c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:54.274498 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55dcc2-3778-4e75-9002-6a5c2d8126c5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:54.274783 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55dcc2-3778-4e75-9002-6a5c2d8126c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:54.275064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55dcc2-3778-4e75-9002-6a5c2d8126c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:54.275807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55dcc2-3778-4e75-9002-6a5c2d8126c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:54.276117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55dcc2-3778-4e75-9002-6a5c2d8126c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:54.278102 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55dcc2-3778-4e75-9002-6a5c2d8126c5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:54.287559 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:54.298098 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 314/626] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:54.298098 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55dcc2-3778-4e75-9002-6a5c2d8126c5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:54.317080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2d48c-42b2-4c9f-bdf3-8229f9546922 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:54.371165 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd02e3b-c995-48bf-a0e4-22c7b4298559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:54.372136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd02e3b-c995-48bf-a0e4-22c7b4298559 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:54.372424 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd02e3b-c995-48bf-a0e4-22c7b4298559 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:54.372704 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd02e3b-c995-48bf-a0e4-22c7b4298559 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:54.380852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd02e3b-c995-48bf-a0e4-22c7b4298559 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:54.382392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd02e3b-c995-48bf-a0e4-22c7b4298559 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:54.405643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-7cd02e3b-c995-48bf-a0e4-22c7b4298559 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:54.405643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:54.405643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:54.405643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:54.405643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:54.405643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:54.405643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:54.405643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:54.405643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:54.405643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:54.405643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:54.405643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:54.405643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:16:54.405643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:16:54.405643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:16:54.406652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:16:54.406652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:54.406652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:54.406652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:54.406652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:54.406652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:54.406652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:54.406652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:54.406652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:54.406652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:54.406652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:54.406652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:54.406652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:54.406652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:54.406652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:54.406652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:54.406652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:54.407752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:16:54.407752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:16:54.407752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:16:54.407752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:16:54.407752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:16:54.407752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:16:54.407752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:54.407752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:54.411867 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 313/627] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:54.413398 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e221ce6-620c-44d5-ab60-c6d7683f141b None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:54.413690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e221ce6-620c-44d5-ab60-c6d7683f141b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:54.413976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e221ce6-620c-44d5-ab60-c6d7683f141b None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:54.415584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 314/628] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:54.421114 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e221ce6-620c-44d5-ab60-c6d7683f141b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:54.482672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55dcc2-3778-4e75-9002-6a5c2d8126c5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:54.533006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:54.542323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d4b2c-04d0-4ad5-bc69-ea45ab3d81d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:54.543385 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4b2c-04d0-4ad5-bc69-ea45ab3d81d5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:54.543909 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4b2c-04d0-4ad5-bc69-ea45ab3d81d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:54.544396 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4b2c-04d0-4ad5-bc69-ea45ab3d81d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:54.545723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d4b2c-04d0-4ad5-bc69-ea45ab3d81d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:54.546196 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d4b2c-04d0-4ad5-bc69-ea45ab3d81d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:54.548567 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d4b2c-04d0-4ad5-bc69-ea45ab3d81d5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:54.569026 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 315/629] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:16:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:54.570291 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d4b2c-04d0-4ad5-bc69-ea45ab3d81d5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:54.579737 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e221ce6-620c-44d5-ab60-c6d7683f141b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:54.638452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71be024-cc2f-4327-9bfd-f46677ddf0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:54.639709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71be024-cc2f-4327-9bfd-f46677ddf0cc None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:54.640154 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71be024-cc2f-4327-9bfd-f46677ddf0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:54.640441 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71be024-cc2f-4327-9bfd-f46677ddf0cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:54.653717 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71be024-cc2f-4327-9bfd-f46677ddf0cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:54.656460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71be024-cc2f-4327-9bfd-f46677ddf0cc None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:54.662769 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:54.686757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-e71be024-cc2f-4327-9bfd-f46677ddf0cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 06:16:54.686757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:54.686757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:54.686757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:54.686757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:54.686757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:54.686757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:54.686757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:54.686757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:54.686757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:54.686757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:54.686757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:54.686757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:16:54.686757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:16:54.686757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:16:54.686757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:54.688651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:54.690640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:54.690640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:54.690640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:16:54.690640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:16:54.690640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:16:54.690640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:16:54.690640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:16:54.690640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:16:54.690640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 06:16:54.690640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:54.690640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 315/630] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 06:16:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:54.690640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbb337-8080-45c1-8679-54f9c77e3f69 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:54.693302 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbb337-8080-45c1-8679-54f9c77e3f69 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:54.693302 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbb337-8080-45c1-8679-54f9c77e3f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:54.719188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3ebbb337-8080-45c1-8679-54f9c77e3f69 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:54.720514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3ebbb337-8080-45c1-8679-54f9c77e3f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:54.763925 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d4b2c-04d0-4ad5-bc69-ea45ab3d81d5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:54.817775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7dc8a-8046-4bd7-a728-7d3d28e67d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:54.818476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7dc8a-8046-4bd7-a728-7d3d28e67d8b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:54.818771 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7dc8a-8046-4bd7-a728-7d3d28e67d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:54.819048 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7dc8a-8046-4bd7-a728-7d3d28e67d8b None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:54.819893 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7dc8a-8046-4bd7-a728-7d3d28e67d8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:54.820878 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7dc8a-8046-4bd7-a728-7d3d28e67d8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:54.821763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7dc8a-8046-4bd7-a728-7d3d28e67d8b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:54.845033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 316/631] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:16:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:54.845033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7dc8a-8046-4bd7-a728-7d3d28e67d8b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:54.872120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 316/632] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:54.873688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbb337-8080-45c1-8679-54f9c77e3f69 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:54.938962 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:55.018068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7dc8a-8046-4bd7-a728-7d3d28e67d8b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:55.047301 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbb337-8080-45c1-8679-54f9c77e3f69 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:55.062338 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946439b-26d5-4b29-a883-3a7a9fceda92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:55.063195 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946439b-26d5-4b29-a883-3a7a9fceda92 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:55.063485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946439b-26d5-4b29-a883-3a7a9fceda92 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:55.063764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946439b-26d5-4b29-a883-3a7a9fceda92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:55.076786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946439b-26d5-4b29-a883-3a7a9fceda92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:55.078371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946439b-26d5-4b29-a883-3a7a9fceda92 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:55.091743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:55.098475 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5386b0-5194-4207-9180-27afe22bd8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:55.099175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5386b0-5194-4207-9180-27afe22bd8cd None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:55.099463 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5386b0-5194-4207-9180-27afe22bd8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:55.099746 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5386b0-5194-4207-9180-27afe22bd8cd None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:55.100621 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5386b0-5194-4207-9180-27afe22bd8cd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:55.100937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5386b0-5194-4207-9180-27afe22bd8cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:55.102503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5386b0-5194-4207-9180-27afe22bd8cd None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:55.105554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-2946439b-26d5-4b29-a883-3a7a9fceda92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:55.105554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:55.105554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:55.105554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:55.105554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:55.105554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:55.105554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:55.105554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:55.105554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:55.105554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:55.105554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:55.105554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:55.105554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:55.105554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:55.105554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:55.107244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:55.107244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:55.107244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:55.107244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:55.107244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:55.107244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:55.107244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:55.107244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:55.107244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:55.107244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:55.107244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:55.107244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:55.107244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:55.107244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:55.107244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:55.107244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:55.107244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:55.109232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:55.109232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:55.109232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:55.109232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:55.109232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 317/633] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:55.109232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946439b-26d5-4b29-a883-3a7a9fceda92 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:55.129201 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 317/634] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 06:16:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:55.130205 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5386b0-5194-4207-9180-27afe22bd8cd None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:55.248416 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946439b-26d5-4b29-a883-3a7a9fceda92 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:55.252833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:55.270965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5386b0-5194-4207-9180-27afe22bd8cd None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:55.299659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddf5bd-4a03-424f-9a83-b4401afd558f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:55.300660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddf5bd-4a03-424f-9a83-b4401afd558f None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:55.301026 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddf5bd-4a03-424f-9a83-b4401afd558f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:55.301434 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddf5bd-4a03-424f-9a83-b4401afd558f None admin] PATH_INFO: `/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/5fc205d053d54506bb6c9227e4a1b044/roles/5cc9ca1bc48b4d749d21d7111b14e187` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:55.318950 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddf5bd-4a03-424f-9a83-b4401afd558f None admin] RBAC: Authorizing `identity:create_grant(project_id=a3d399dc95424b848b736c0d0298d9a9, user_id=5fc205d053d54506bb6c9227e4a1b044, role_id=5cc9ca1bc48b4d749d21d7111b14e187)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:55.323444 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddf5bd-4a03-424f-9a83-b4401afd558f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:55.324170 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26182a4e-e072-4a91-b8c5-7054d06d4ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:55.325449 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26182a4e-e072-4a91-b8c5-7054d06d4ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:55.325944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26182a4e-e072-4a91-b8c5-7054d06d4ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:55.326437 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26182a4e-e072-4a91-b8c5-7054d06d4ab5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:55.342439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26182a4e-e072-4a91-b8c5-7054d06d4ab5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:55.342779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 318/635] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:16:55 2022] PUT /identity/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/5fc205d053d54506bb6c9227e4a1b044/roles/5cc9ca1bc48b4d749d21d7111b14e187 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:16:55.345174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c38d6-6185-4028-9f15-f122a10cba55 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:55.345729 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c38d6-6185-4028-9f15-f122a10cba55 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:55.346227 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c38d6-6185-4028-9f15-f122a10cba55 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:55.349103 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 319/636] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:55.350323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c38d6-6185-4028-9f15-f122a10cba55 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:55.352075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-26182a4e-e072-4a91-b8c5-7054d06d4ab5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:55.352075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:55.352075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:55.352075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:55.352075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:55.352075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:55.352075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:55.352075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:55.352075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:55.352075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:55.352075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:55.352075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:55.352075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:55.352075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:55.352075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:55.352075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:55.354062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:55.354062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:55.354062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:55.354062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:55.354062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:55.354062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:55.354062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:55.354062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:55.354062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:55.354062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:55.354062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:55.354062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:55.354062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:55.354062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:55.354062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:55.354062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:55.358274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:55.358274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:55.358274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:55.358274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:55.358274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:55.358274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:55.358274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:55.358274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:55.358274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:55.358274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:55.358274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:55.358274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:55.358274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 318/637] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:55.358274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26182a4e-e072-4a91-b8c5-7054d06d4ab5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:55.453409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:55.546352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26182a4e-e072-4a91-b8c5-7054d06d4ab5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:55.551424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c38d6-6185-4028-9f15-f122a10cba55 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:55.609013 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de2efa-82f5-40bd-a998-e7595f17e173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:55.610127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de2efa-82f5-40bd-a998-e7595f17e173 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:55.610553 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de2efa-82f5-40bd-a998-e7595f17e173 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:55.610853 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf25a11-b547-4f0b-822b-487e4267bab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:55.611030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de2efa-82f5-40bd-a998-e7595f17e173 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:55.611974 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf25a11-b547-4f0b-822b-487e4267bab4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:55.612244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de2efa-82f5-40bd-a998-e7595f17e173 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:55.612479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf25a11-b547-4f0b-822b-487e4267bab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:55.612770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de2efa-82f5-40bd-a998-e7595f17e173 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:55.612980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf25a11-b547-4f0b-822b-487e4267bab4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:55.614900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de2efa-82f5-40bd-a998-e7595f17e173 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:55.640678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 320/638] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:55.640678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2faac1-0305-41e5-a151-98a0654ae427 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:55.640678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2faac1-0305-41e5-a151-98a0654ae427 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:55.640678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2faac1-0305-41e5-a151-98a0654ae427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:55.668541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-7a2faac1-0305-41e5-a151-98a0654ae427 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:55.670564 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7a2faac1-0305-41e5-a151-98a0654ae427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:55.688760 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:55.738212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf25a11-b547-4f0b-822b-487e4267bab4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:55.740900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf25a11-b547-4f0b-822b-487e4267bab4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:55.801387 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 321/639] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:55.801387 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2faac1-0305-41e5-a151-98a0654ae427 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:55.817720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:55.885331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 319/640] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:55.887231 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf25a11-b547-4f0b-822b-487e4267bab4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:55.939432 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2faac1-0305-41e5-a151-98a0654ae427 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:55.977068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee76e84-491f-4bc7-bc95-418e5e6ceecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:55.977777 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee76e84-491f-4bc7-bc95-418e5e6ceecf None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:55.978056 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee76e84-491f-4bc7-bc95-418e5e6ceecf None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:55.978330 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee76e84-491f-4bc7-bc95-418e5e6ceecf None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:55.979125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee76e84-491f-4bc7-bc95-418e5e6ceecf None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:55.979435 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee76e84-491f-4bc7-bc95-418e5e6ceecf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:55.981940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee76e84-491f-4bc7-bc95-418e5e6ceecf None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:55.996154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 322/641] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:16:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:55.996154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee76e84-491f-4bc7-bc95-418e5e6ceecf None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:56.003837 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:56.067765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf25a11-b547-4f0b-822b-487e4267bab4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:56.115795 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d3b51-ac17-40dc-931f-aa1d6d4b9d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:56.117393 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d3b51-ac17-40dc-931f-aa1d6d4b9d7f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:56.118056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d3b51-ac17-40dc-931f-aa1d6d4b9d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:56.118527 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d3b51-ac17-40dc-931f-aa1d6d4b9d7f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:56.128049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d3b51-ac17-40dc-931f-aa1d6d4b9d7f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:56.130127 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d3b51-ac17-40dc-931f-aa1d6d4b9d7f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:56.143990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee76e84-491f-4bc7-bc95-418e5e6ceecf None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:56.149057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-8b4d3b51-ac17-40dc-931f-aa1d6d4b9d7f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 06:16:56.149057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:56.149057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:56.149057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:56.149057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:56.149057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:56.149057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:56.149057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:56.149057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:56.149057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:56.149057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:56.149057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:56.149057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:16:56.149057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:16:56.149057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:16:56.149057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:56.150218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:56.151359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:56.151359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:56.151359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:16:56.151359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:16:56.151359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:16:56.151359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:16:56.151359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:16:56.151359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:16:56.151359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 06:16:56.151359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:56.153529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 320/642] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:56.154766 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d3b51-ac17-40dc-931f-aa1d6d4b9d7f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:56.172994 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:56.210899 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68961094-d9b4-49c3-88b1-f144de60e4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:56.212144 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68961094-d9b4-49c3-88b1-f144de60e4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:56.212598 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68961094-d9b4-49c3-88b1-f144de60e4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:56.213090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68961094-d9b4-49c3-88b1-f144de60e4d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:56.232477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68961094-d9b4-49c3-88b1-f144de60e4d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:56.240346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-68961094-d9b4-49c3-88b1-f144de60e4d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:56.240346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:56.240346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:56.240346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:56.240346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:56.240346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:56.240346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:56.240346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:56.240346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:56.240346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:56.240346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:56.240346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:56.240346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:56.240346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:56.240346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:56.240346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:56.242187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:56.242187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:56.242187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:56.242187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:56.242187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:56.242187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:56.242187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:56.242187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:56.242187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:56.242187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:56.242187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:56.242187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:56.242187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:56.242187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:56.242187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:56.242187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:56.243912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:56.243912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:56.243912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:56.243912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:56.243912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:56.243912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:56.243912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:56.243912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:56.243912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:56.243912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:56.243912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:56.243912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:56.243912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 323/643] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:56.243912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68961094-d9b4-49c3-88b1-f144de60e4d4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:56.314051 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d3b51-ac17-40dc-931f-aa1d6d4b9d7f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:56.322260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:56.365978 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68961094-d9b4-49c3-88b1-f144de60e4d4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:56.378308 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0155d-7d35-41f9-b692-6c8070077b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:56.379455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0155d-7d35-41f9-b692-6c8070077b97 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:56.379920 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0155d-7d35-41f9-b692-6c8070077b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:56.380374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0155d-7d35-41f9-b692-6c8070077b97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:56.397226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0155d-7d35-41f9-b692-6c8070077b97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:56.400149 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0155d-7d35-41f9-b692-6c8070077b97 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:56.420756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fcd0c-1237-4952-85dc-ecbe1312eb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:56.422039 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fcd0c-1237-4952-85dc-ecbe1312eb8f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:56.422499 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fcd0c-1237-4952-85dc-ecbe1312eb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:56.422948 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fcd0c-1237-4952-85dc-ecbe1312eb8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:56.427682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-12c0155d-7d35-41f9-b692-6c8070077b97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:56.427682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:56.427682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:56.427682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:56.427682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:56.427682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:56.427682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:56.427682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:56.427682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:56.427682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:56.427682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:56.427682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:56.427682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:56.427682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:56.427682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:56.428617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:56.428617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:56.428617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:56.428617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:56.428617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:56.428617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:56.428617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:56.428617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:56.428617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:56.428617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:56.428617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:56.428617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:56.428617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:56.428617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:56.428617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:56.428617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:56.428617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:56.429713 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:56.429713 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:56.429713 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:56.429713 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:56.436343 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 321/644] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:56.436343 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0155d-7d35-41f9-b692-6c8070077b97 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:56.443486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fcd0c-1237-4952-85dc-ecbe1312eb8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:56.446183 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fcd0c-1237-4952-85dc-ecbe1312eb8f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:56.470481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-274fcd0c-1237-4952-85dc-ecbe1312eb8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:56.470481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:56.470481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:56.470481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:56.470481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:56.470481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:56.470481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:56.470481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:56.470481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:56.470481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:56.470481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:56.470481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:56.470481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:56.470481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:56.470481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:56.472260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:56.472260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:56.472260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:56.472260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:56.472260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:56.472260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:56.472260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:56.472260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:56.472260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:56.472260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:56.472260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:56.472260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:56.472260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:56.472260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:56.472260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:56.472260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:56.472260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:56.474046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:56.474046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:56.474046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:56.474046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:56.474046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 324/645] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:56.475300 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fcd0c-1237-4952-85dc-ecbe1312eb8f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:56.547153 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:56.630899 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0155d-7d35-41f9-b692-6c8070077b97 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:56.669596 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fcd0c-1237-4952-85dc-ecbe1312eb8f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:56.695555 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37d350-1f86-4ee8-b2ca-e1752f549ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:56.696725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37d350-1f86-4ee8-b2ca-e1752f549ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:56.697235 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37d350-1f86-4ee8-b2ca-e1752f549ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:56.697797 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37d350-1f86-4ee8-b2ca-e1752f549ad0 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:56.699717 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37d350-1f86-4ee8-b2ca-e1752f549ad0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:56.700287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37d350-1f86-4ee8-b2ca-e1752f549ad0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:56.700533 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:56.703122 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37d350-1f86-4ee8-b2ca-e1752f549ad0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:56.733349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb02316-b58a-4165-8e33-1c872200a3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:56.734526 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb02316-b58a-4165-8e33-1c872200a3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:56.735016 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb02316-b58a-4165-8e33-1c872200a3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:56.735513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb02316-b58a-4165-8e33-1c872200a3b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:56.736943 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb02316-b58a-4165-8e33-1c872200a3b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:56.737524 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb02316-b58a-4165-8e33-1c872200a3b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:56.742954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 322/646] 10.222.0.65 () {56 vars in 1137 bytes} [Thu May 19 06:16:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:56.742954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37d350-1f86-4ee8-b2ca-e1752f549ad0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:56.743201 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb02316-b58a-4165-8e33-1c872200a3b0 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:56.760668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 325/647] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:16:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:56.760668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb02316-b58a-4165-8e33-1c872200a3b0 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:56.912060 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37d350-1f86-4ee8-b2ca-e1752f549ad0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:56.927518 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb02316-b58a-4165-8e33-1c872200a3b0 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:56.942392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:56.972640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc47744-496c-4b9f-b13e-1ab129f9b038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:56.973560 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc47744-496c-4b9f-b13e-1ab129f9b038 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:56.974024 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc47744-496c-4b9f-b13e-1ab129f9b038 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:56.974458 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc47744-496c-4b9f-b13e-1ab129f9b038 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:56.975666 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc47744-496c-4b9f-b13e-1ab129f9b038 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:56.976175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc47744-496c-4b9f-b13e-1ab129f9b038 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:56.978413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc47744-496c-4b9f-b13e-1ab129f9b038 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:56.987418 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e81a0-97fd-4620-bdfc-a624dc718c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:56.988579 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e81a0-97fd-4620-bdfc-a624dc718c71 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:56.989109 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e81a0-97fd-4620-bdfc-a624dc718c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:56.989692 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e81a0-97fd-4620-bdfc-a624dc718c71 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:56.991098 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e81a0-97fd-4620-bdfc-a624dc718c71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:56.991659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e81a0-97fd-4620-bdfc-a624dc718c71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:56.994405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e81a0-97fd-4620-bdfc-a624dc718c71 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:57.001368 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 323/648] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:57.001368 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc47744-496c-4b9f-b13e-1ab129f9b038 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:57.029671 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 326/649] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:57.031425 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e81a0-97fd-4620-bdfc-a624dc718c71 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:57.116111 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:57.131919 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc47744-496c-4b9f-b13e-1ab129f9b038 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:57.189579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f8c69-4ed2-4cdf-a236-c853ad8d511e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:57.190134 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f8c69-4ed2-4cdf-a236-c853ad8d511e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:57.190422 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f8c69-4ed2-4cdf-a236-c853ad8d511e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:57.190721 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f8c69-4ed2-4cdf-a236-c853ad8d511e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:57.227359 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e81a0-97fd-4620-bdfc-a624dc718c71 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:57.252806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:57.287797 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f8c69-4ed2-4cdf-a236-c853ad8d511e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:57.289748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f8c69-4ed2-4cdf-a236-c853ad8d511e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:57.295669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17187c8-4a8b-4cdc-b22c-963035bd2f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:57.297042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17187c8-4a8b-4cdc-b22c-963035bd2f76 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:57.297569 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17187c8-4a8b-4cdc-b22c-963035bd2f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:57.298072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17187c8-4a8b-4cdc-b22c-963035bd2f76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:57.322366 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17187c8-4a8b-4cdc-b22c-963035bd2f76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:57.322366 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17187c8-4a8b-4cdc-b22c-963035bd2f76 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:57.356454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-c17187c8-4a8b-4cdc-b22c-963035bd2f76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:57.356454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:57.356454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:57.356454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:57.356454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:57.356454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:57.356454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:57.356454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:57.356454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:57.356454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:57.356454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:57.356454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:57.356454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:57.356454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:57.356454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:57.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:57.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:57.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:57.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:57.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:57.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:57.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:57.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:57.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:57.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:57.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:57.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:57.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:57.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:57.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:57.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:57.357546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:57.358447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:57.358447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:57.358447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:57.358447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:57.365892 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 327/650] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:57.367750 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17187c8-4a8b-4cdc-b22c-963035bd2f76 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:57.440312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 324/651] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:57.441722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f8c69-4ed2-4cdf-a236-c853ad8d511e None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:57.538863 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:57.553075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:57.558198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17187c8-4a8b-4cdc-b22c-963035bd2f76 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:57.588186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f8c69-4ed2-4cdf-a236-c853ad8d511e None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:57.619750 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7492572-b8e0-4e9e-9e7a-fabb74f4f702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:57.621155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7492572-b8e0-4e9e-9e7a-fabb74f4f702 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:57.621477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7492572-b8e0-4e9e-9e7a-fabb74f4f702 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:57.621739 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7492572-b8e0-4e9e-9e7a-fabb74f4f702 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:57.634123 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7492572-b8e0-4e9e-9e7a-fabb74f4f702 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:57.637451 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7492572-b8e0-4e9e-9e7a-fabb74f4f702 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:57.651040 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ff68c-b284-419d-9813-d3d810f1808e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:57.651896 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ff68c-b284-419d-9813-d3d810f1808e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:57.652177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ff68c-b284-419d-9813-d3d810f1808e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:57.652450 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ff68c-b284-419d-9813-d3d810f1808e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:57.658449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-c7492572-b8e0-4e9e-9e7a-fabb74f4f702 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:57.658449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:57.658449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:57.658449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:57.658449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:57.658449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:57.658449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:57.658449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:57.658449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:57.658449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:57.658449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:57.658449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:57.658449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:16:57.658449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:16:57.658449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:16:57.660142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:16:57.660142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:57.660142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:57.660142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:57.660142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:57.660142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:57.660142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:57.660142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:57.660142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:57.660142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:57.660142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:57.660142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:57.660142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:57.660142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:57.660142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:57.660142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:57.660142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:57.661955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:16:57.661955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:16:57.661955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:16:57.661955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:16:57.661955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:16:57.661955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:16:57.661955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:57.661955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:57.664523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 328/652] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:57.665307 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ff68c-b284-419d-9813-d3d810f1808e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:57.665531 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7492572-b8e0-4e9e-9e7a-fabb74f4f702 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:57.667079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ff68c-b284-419d-9813-d3d810f1808e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:57.686894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-8f9ff68c-b284-419d-9813-d3d810f1808e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 06:16:57.686894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:57.686894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:57.686894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:57.686894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:57.686894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:57.686894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:57.686894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:57.686894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:57.686894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:57.686894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:57.686894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:57.686894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:16:57.686894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:16:57.686894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:16:57.686894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:57.688750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:57.691284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:57.691284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:57.691284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:16:57.691284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:16:57.691284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:16:57.691284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:16:57.691284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:16:57.691284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:16:57.691284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 06:16:57.691284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:57.691284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 325/653] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:16:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:57.691284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ff68c-b284-419d-9813-d3d810f1808e None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:57.816691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7492572-b8e0-4e9e-9e7a-fabb74f4f702 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:57.850869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ff68c-b284-419d-9813-d3d810f1808e None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:57.859298 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb02a7-e48c-4f25-9f6b-73c2dfa0b83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:57.859964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb02a7-e48c-4f25-9f6b-73c2dfa0b83c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:57.860250 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb02a7-e48c-4f25-9f6b-73c2dfa0b83c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:57.860520 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb02a7-e48c-4f25-9f6b-73c2dfa0b83c None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:57.861255 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb02a7-e48c-4f25-9f6b-73c2dfa0b83c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:57.861619 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb02a7-e48c-4f25-9f6b-73c2dfa0b83c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:57.863018 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb02a7-e48c-4f25-9f6b-73c2dfa0b83c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:57.878993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 329/654] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:57.879710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb02a7-e48c-4f25-9f6b-73c2dfa0b83c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:57.895905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:57.918479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e1555-d210-4ac7-bd95-0a406a493b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:57.919450 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e1555-d210-4ac7-bd95-0a406a493b77 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:57.919968 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e1555-d210-4ac7-bd95-0a406a493b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:57.920308 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e1555-d210-4ac7-bd95-0a406a493b77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:57.928431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:57.934135 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e1555-d210-4ac7-bd95-0a406a493b77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:57.935624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e1555-d210-4ac7-bd95-0a406a493b77 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:57.953863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-a85e1555-d210-4ac7-bd95-0a406a493b77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:57.953863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:57.953863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:57.953863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:57.953863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:57.953863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:57.953863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:57.953863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:57.953863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:57.953863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:57.953863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:57.953863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:57.953863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:57.953863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:57.953863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:57.955010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:57.955010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:57.955010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:57.955010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:57.955010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:57.955010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:57.955010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:57.955010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:57.955010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:57.955010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:57.955010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:57.955010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:57.955010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:57.955010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:57.955010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:57.955010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:57.955010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:57.956353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:57.956353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:57.956353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:57.956353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:57.956353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 326/655] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:57.956353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e1555-d210-4ac7-bd95-0a406a493b77 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:57.998662 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb02a7-e48c-4f25-9f6b-73c2dfa0b83c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:58.038102 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86f017-ba2a-477c-93ca-5670c40b6f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:58.038774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86f017-ba2a-477c-93ca-5670c40b6f52 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:58.039063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86f017-ba2a-477c-93ca-5670c40b6f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:58.039341 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86f017-ba2a-477c-93ca-5670c40b6f52 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:58.040140 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86f017-ba2a-477c-93ca-5670c40b6f52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:58.040453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86f017-ba2a-477c-93ca-5670c40b6f52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:58.041944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86f017-ba2a-477c-93ca-5670c40b6f52 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:58.056758 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 330/656] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:16:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:58.057656 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86f017-ba2a-477c-93ca-5670c40b6f52 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:58.079889 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e1555-d210-4ac7-bd95-0a406a493b77 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:58.118641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda11b02-f156-4e85-aaf8-a28682986e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:58.119330 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda11b02-f156-4e85-aaf8-a28682986e38 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:58.119613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda11b02-f156-4e85-aaf8-a28682986e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:58.119886 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda11b02-f156-4e85-aaf8-a28682986e38 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:58.120780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda11b02-f156-4e85-aaf8-a28682986e38 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:58.121552 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda11b02-f156-4e85-aaf8-a28682986e38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:58.122553 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda11b02-f156-4e85-aaf8-a28682986e38 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:58.138425 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:58.141435 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 327/657] 10.222.0.65 () {56 vars in 1137 bytes} [Thu May 19 06:16:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:58.142755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0241f5-d1bd-4dbe-8eb0-b1444291d832 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:58.143041 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0241f5-d1bd-4dbe-8eb0-b1444291d832 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:58.143314 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0241f5-d1bd-4dbe-8eb0-b1444291d832 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:58.144630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 328/658] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:16:58.147582 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0241f5-d1bd-4dbe-8eb0-b1444291d832 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:58.175805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86f017-ba2a-477c-93ca-5670c40b6f52 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:58.211372 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:58.214789 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdb32e-b55e-41ad-8bde-ecfcd90a0ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:58.215672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdb32e-b55e-41ad-8bde-ecfcd90a0ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:58.215969 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdb32e-b55e-41ad-8bde-ecfcd90a0ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:58.216252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdb32e-b55e-41ad-8bde-ecfcd90a0ac2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:58.225205 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdb32e-b55e-41ad-8bde-ecfcd90a0ac2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:58.226757 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdb32e-b55e-41ad-8bde-ecfcd90a0ac2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:58.247453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-cbbdb32e-b55e-41ad-8bde-ecfcd90a0ac2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 06:16:58.247453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:58.247453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:58.247453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:58.247453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:58.247453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:58.247453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:58.247453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:58.247453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:58.247453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:58.247453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:58.247453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:58.247453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:16:58.247453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:16:58.247453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:16:58.247453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:58.254327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:58.255304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:58.255304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:58.255304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:16:58.255304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:16:58.255304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:16:58.255304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:16:58.255304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:16:58.255304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:16:58.255304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 06:16:58.255304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:58.255304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 331/659] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:58.255304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdb32e-b55e-41ad-8bde-ecfcd90a0ac2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:58.302622 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0241f5-d1bd-4dbe-8eb0-b1444291d832 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:58.350015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46c089-51f5-416e-be93-011068dbb631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:58.350859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46c089-51f5-416e-be93-011068dbb631 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:58.351275 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46c089-51f5-416e-be93-011068dbb631 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:58.351734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46c089-51f5-416e-be93-011068dbb631 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:58.352618 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46c089-51f5-416e-be93-011068dbb631 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:58.353152 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46c089-51f5-416e-be93-011068dbb631 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:58.355309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46c089-51f5-416e-be93-011068dbb631 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:58.369373 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 329/660] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:58.371165 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46c089-51f5-416e-be93-011068dbb631 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:58.374968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdb32e-b55e-41ad-8bde-ecfcd90a0ac2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:58.414507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:58.421749 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb0e15-1abd-4952-af4f-332a3e328258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:58.422414 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb0e15-1abd-4952-af4f-332a3e328258 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:58.422743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb0e15-1abd-4952-af4f-332a3e328258 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:58.423028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb0e15-1abd-4952-af4f-332a3e328258 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:58.423875 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb0e15-1abd-4952-af4f-332a3e328258 None admin] RBAC: Adding query filter params (role.id=5cc9ca1bc48b4d749d21d7111b14e187, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=2a09f4878eac4d60a9a7dcedba15c05d) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:58.424189 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb0e15-1abd-4952-af4f-332a3e328258 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:58.425691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb0e15-1abd-4952-af4f-332a3e328258 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:58.432051 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 332/661] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:16:58 2022] GET /identity/v3/role_assignments?user.id=2a09f4878eac4d60a9a7dcedba15c05d&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=5cc9ca1bc48b4d749d21d7111b14e187 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:58.433846 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8b97a-dd10-4870-90aa-b3a9273cb861 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:58.434141 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8b97a-dd10-4870-90aa-b3a9273cb861 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:58.434423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8b97a-dd10-4870-90aa-b3a9273cb861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:58.455033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-7db8b97a-dd10-4870-90aa-b3a9273cb861 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:16:58.456328 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7db8b97a-dd10-4870-90aa-b3a9273cb861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:16:58.510032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46c089-51f5-416e-be93-011068dbb631 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:58.530286 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:58.559946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 333/662] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:16:58.560956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8b97a-dd10-4870-90aa-b3a9273cb861 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:58.588760 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf95735-07cf-42cb-822f-222fe69bebb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:58.590814 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf95735-07cf-42cb-822f-222fe69bebb2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:58.591007 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf95735-07cf-42cb-822f-222fe69bebb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:58.591144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf95735-07cf-42cb-822f-222fe69bebb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:58.711992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf95735-07cf-42cb-822f-222fe69bebb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:58.713917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf95735-07cf-42cb-822f-222fe69bebb2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:58.715253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8b97a-dd10-4870-90aa-b3a9273cb861 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:58.753096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:58.767571 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f815a-6423-4e13-a1f5-2de90cbab0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:58.768178 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f815a-6423-4e13-a1f5-2de90cbab0bb None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:58.768445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f815a-6423-4e13-a1f5-2de90cbab0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:58.768731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f815a-6423-4e13-a1f5-2de90cbab0bb None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:58.769619 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f815a-6423-4e13-a1f5-2de90cbab0bb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:58.769937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f815a-6423-4e13-a1f5-2de90cbab0bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:58.772740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f815a-6423-4e13-a1f5-2de90cbab0bb None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:58.794853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 334/663] 10.222.0.65 () {56 vars in 1139 bytes} [Thu May 19 06:16:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:58.795729 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f815a-6423-4e13-a1f5-2de90cbab0bb None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:58.870759 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 330/664] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:58.872597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf95735-07cf-42cb-822f-222fe69bebb2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:58.895850 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:58.946040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f815a-6423-4e13-a1f5-2de90cbab0bb None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:58.993713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633c085-6fd6-422c-bf49-4cb40896146e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:58.994514 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633c085-6fd6-422c-bf49-4cb40896146e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:58.994795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633c085-6fd6-422c-bf49-4cb40896146e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:58.996249 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633c085-6fd6-422c-bf49-4cb40896146e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:59.006587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633c085-6fd6-422c-bf49-4cb40896146e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:59.008207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633c085-6fd6-422c-bf49-4cb40896146e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:59.026968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-0633c085-6fd6-422c-bf49-4cb40896146e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:59.026968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:59.026968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:59.026968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:59.026968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:59.026968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:59.026968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:59.026968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:59.026968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:59.026968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:59.026968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:59.026968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:59.026968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:16:59.026968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:16:59.026968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:16:59.028506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:16:59.028506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:59.028506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:59.028506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:59.028506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:59.028506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:59.028506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:59.028506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:59.028506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:16:59.028506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:59.028506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:59.028506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:59.028506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:59.028506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:59.028506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:59.028506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:59.028506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:59.030383 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:16:59.030383 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:16:59.030383 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:16:59.030383 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:16:59.030383 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:16:59.030383 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:16:59.030383 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:16:59.030383 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:16:59.033621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 335/665] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:16:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:59.034395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633c085-6fd6-422c-bf49-4cb40896146e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:59.056626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf95735-07cf-42cb-822f-222fe69bebb2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:59.102461 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c47d3d-5ba7-425b-8930-b78693c8b752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:59.103298 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c47d3d-5ba7-425b-8930-b78693c8b752 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:59.103909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c47d3d-5ba7-425b-8930-b78693c8b752 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:59.104199 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c47d3d-5ba7-425b-8930-b78693c8b752 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:59.108022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:59.117674 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c47d3d-5ba7-425b-8930-b78693c8b752 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:16:59.125861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-b3c47d3d-5ba7-425b-8930-b78693c8b752 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:59.125861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:59.125861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:59.125861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:59.125861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:59.125861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:59.125861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:59.125861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:59.125861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:59.125861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:59.125861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:59.125861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:59.125861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:16:59.125861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:16:59.125861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:16:59.125861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:16:59.133374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:16:59.133374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:16:59.133374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:16:59.133374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:16:59.133374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:59.133374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:59.133374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:59.133374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:59.133374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:59.133374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:59.133374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:59.133374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:59.133374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:59.133374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:59.133374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:59.133374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:59.134662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:59.134662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:59.134662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:59.134662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:59.134662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:16:59.134662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:16:59.134662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:16:59.134662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:16:59.134662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:16:59.134662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:16:59.134662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:16:59.134662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:59.134662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 331/666] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:16:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:59.134662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c47d3d-5ba7-425b-8930-b78693c8b752 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:59.202236 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633c085-6fd6-422c-bf49-4cb40896146e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:59.248476 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:59.280069 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9b582-49e0-4e28-a149-a6b3a8565ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:59.281408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9b582-49e0-4e28-a149-a6b3a8565ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:59.281995 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9b582-49e0-4e28-a149-a6b3a8565ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:59.282570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9b582-49e0-4e28-a149-a6b3a8565ed1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:59.310721 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c47d3d-5ba7-425b-8930-b78693c8b752 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:59.374892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bcc5c-3fb2-4cb0-88b5-d05589168a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:59.376548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bcc5c-3fb2-4cb0-88b5-d05589168a76 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:59.377089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bcc5c-3fb2-4cb0-88b5-d05589168a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:59.377690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bcc5c-3fb2-4cb0-88b5-d05589168a76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:59.395094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bcc5c-3fb2-4cb0-88b5-d05589168a76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:59.396737 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bcc5c-3fb2-4cb0-88b5-d05589168a76 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:59.406985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9b582-49e0-4e28-a149-a6b3a8565ed1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:59.409071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9b582-49e0-4e28-a149-a6b3a8565ed1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:59.418935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-f68bcc5c-3fb2-4cb0-88b5-d05589168a76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:59.418935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:16:59.418935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:16:59.418935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:16:59.418935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:16:59.418935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:16:59.418935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:16:59.418935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:16:59.418935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:16:59.418935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:16:59.418935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:16:59.418935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:16:59.418935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:16:59.418935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:16:59.418935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:16:59.420960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:16:59.420960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:16:59.420960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:16:59.420960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:16:59.420960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:16:59.420960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:16:59.420960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:16:59.420960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:16:59.420960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:16:59.420960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:16:59.420960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:16:59.420960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:16:59.420960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:16:59.420960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:16:59.420960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:16:59.420960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:16:59.420960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:16:59.422496 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:16:59.422496 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:16:59.422496 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:16:59.422496 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:16:59.422496 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 332/667] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:16:59.430280 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bcc5c-3fb2-4cb0-88b5-d05589168a76 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:59.497327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:59.608787 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 336/668] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:16:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:16:59.610280 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9b582-49e0-4e28-a149-a6b3a8565ed1 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:59.632283 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bcc5c-3fb2-4cb0-88b5-d05589168a76 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:59.672807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:59.701985 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571b3af-1ec2-4921-a8d5-698ad4a550d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:59.702837 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571b3af-1ec2-4921-a8d5-698ad4a550d1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:59.703255 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571b3af-1ec2-4921-a8d5-698ad4a550d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:59.703629 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571b3af-1ec2-4921-a8d5-698ad4a550d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:59.705103 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571b3af-1ec2-4921-a8d5-698ad4a550d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:59.705569 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571b3af-1ec2-4921-a8d5-698ad4a550d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:59.707170 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571b3af-1ec2-4921-a8d5-698ad4a550d1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:59.722298 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 333/669] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:16:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:59.723831 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571b3af-1ec2-4921-a8d5-698ad4a550d1 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:59.764517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9b582-49e0-4e28-a149-a6b3a8565ed1 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:59.822082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1333ef-b73f-4802-9c00-1fad79a75d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:59.823162 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1333ef-b73f-4802-9c00-1fad79a75d4c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:59.824095 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1333ef-b73f-4802-9c00-1fad79a75d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:59.824552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1333ef-b73f-4802-9c00-1fad79a75d4c None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:59.825664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1333ef-b73f-4802-9c00-1fad79a75d4c None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:59.826166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1333ef-b73f-4802-9c00-1fad79a75d4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:59.828589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1333ef-b73f-4802-9c00-1fad79a75d4c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:59.847960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 337/670] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:16:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:59.847960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1333ef-b73f-4802-9c00-1fad79a75d4c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:16:59.863286 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:16:59.900967 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571b3af-1ec2-4921-a8d5-698ad4a550d1 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:16:59.952198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246e50c-0d00-41cd-9171-837f832094f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:16:59.952877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246e50c-0d00-41cd-9171-837f832094f1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:16:59.953164 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246e50c-0d00-41cd-9171-837f832094f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:16:59.953471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246e50c-0d00-41cd-9171-837f832094f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:16:59.954201 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246e50c-0d00-41cd-9171-837f832094f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:16:59.954512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246e50c-0d00-41cd-9171-837f832094f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:16:59.955897 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246e50c-0d00-41cd-9171-837f832094f1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:16:59.972889 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 334/671] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:16:59.973878 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246e50c-0d00-41cd-9171-837f832094f1 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:00.020825 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1333ef-b73f-4802-9c00-1fad79a75d4c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:00.023735 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:00.071705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9416f-d978-48ce-96e6-eaacede1f7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:00.074074 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9416f-d978-48ce-96e6-eaacede1f7c0 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:00.074675 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9416f-d978-48ce-96e6-eaacede1f7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:00.075137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9416f-d978-48ce-96e6-eaacede1f7c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:00.094394 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9416f-d978-48ce-96e6-eaacede1f7c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:00.096652 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9416f-d978-48ce-96e6-eaacede1f7c0 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:00.114261 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246e50c-0d00-41cd-9171-837f832094f1 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:00.126152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-5fa9416f-d978-48ce-96e6-eaacede1f7c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:00.126152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:00.126152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:00.126152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:00.126152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:00.126152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:00.126152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:00.126152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:00.126152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:00.126152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:00.126152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:00.126152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:00.126152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:00.126152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:00.126152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:00.127101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:00.127101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:00.127101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:00.127101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:00.127101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:00.127101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:00.127101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:00.127101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:00.127101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:00.127101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:00.127101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:00.127101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:00.127101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:00.127101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:00.127101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:00.127101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:00.127101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:00.128057 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:00.128057 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:00.128057 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:00.128057 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:00.128057 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 338/672] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:16:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:00.129324 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9416f-d978-48ce-96e6-eaacede1f7c0 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:00.172979 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01f846-4b0e-417f-b11e-3761988bb6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:00.173544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01f846-4b0e-417f-b11e-3761988bb6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:00.173978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01f846-4b0e-417f-b11e-3761988bb6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:00.174441 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01f846-4b0e-417f-b11e-3761988bb6a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:00.175821 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01f846-4b0e-417f-b11e-3761988bb6a7 None admin] RBAC: Adding query filter params (role.id=5cc9ca1bc48b4d749d21d7111b14e187, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=a812c3b9b20f44f8814051381a8005b8) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:00.176352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01f846-4b0e-417f-b11e-3761988bb6a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:00.178854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01f846-4b0e-417f-b11e-3761988bb6a7 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:00.197322 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 335/673] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:16:59 2022] GET /identity/v3/role_assignments?user.id=a812c3b9b20f44f8814051381a8005b8&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=5cc9ca1bc48b4d749d21d7111b14e187 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:00.197322 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01f846-4b0e-417f-b11e-3761988bb6a7 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:00.230344 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:00.318865 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9416f-d978-48ce-96e6-eaacede1f7c0 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:00.392759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37abb7-86e1-4802-bd1f-79d1f0a8ba41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:00.394229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37abb7-86e1-4802-bd1f-79d1f0a8ba41 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:00.394725 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37abb7-86e1-4802-bd1f-79d1f0a8ba41 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:00.395206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37abb7-86e1-4802-bd1f-79d1f0a8ba41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:00.396022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01f846-4b0e-417f-b11e-3761988bb6a7 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:00.399834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:00.413746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37abb7-86e1-4802-bd1f-79d1f0a8ba41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:00.415756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37abb7-86e1-4802-bd1f-79d1f0a8ba41 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:00.438100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-6c37abb7-86e1-4802-bd1f-79d1f0a8ba41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:00.438100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:00.438100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:00.438100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:00.438100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:00.438100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:00.438100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:00.438100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:00.438100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:00.438100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:00.438100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:00.438100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:00.438100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:00.438100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:00.438100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:00.439038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:00.439038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:00.439038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:00.439038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:00.439038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:00.439038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:00.439038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:00.439038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:00.439038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:00.439038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:00.439038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:00.439038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:00.439038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:00.439038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:00.439038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:00.439038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:00.439038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:00.439918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:00.439918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:00.439918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:00.439918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:00.447209 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 339/674] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:00.449186 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37abb7-86e1-4802-bd1f-79d1f0a8ba41 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:00.462184 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414fe09-293d-438c-90db-638fb9ac4582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:00.463467 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414fe09-293d-438c-90db-638fb9ac4582 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:00.463937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414fe09-293d-438c-90db-638fb9ac4582 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:00.464431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414fe09-293d-438c-90db-638fb9ac4582 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:00.482456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414fe09-293d-438c-90db-638fb9ac4582 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:00.484526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414fe09-293d-438c-90db-638fb9ac4582 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:00.514146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-2414fe09-293d-438c-90db-638fb9ac4582 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:00.514146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:00.514146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:00.514146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:00.514146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:00.514146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:00.514146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:00.514146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:00.514146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:00.514146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:00.514146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:00.514146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:00.514146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:00.514146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:00.514146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:00.515761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:00.515761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:00.515761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:00.515761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:00.515761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:00.515761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:00.515761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:00.515761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:00.515761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:00.515761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:00.515761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:00.515761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:00.515761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:00.515761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:00.515761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:00.515761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:00.515761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:00.517474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:00.517474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:00.517474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:00.517474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:00.517474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:00.517474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:00.517474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:00.517474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:00.525339 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 336/675] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:00.525339 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414fe09-293d-438c-90db-638fb9ac4582 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:00.613395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37abb7-86e1-4802-bd1f-79d1f0a8ba41 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:00.634367 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:00.662547 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414fe09-293d-438c-90db-638fb9ac4582 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:00.678223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77a6d4-4172-4d2c-80b5-bb764e9fff72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:00.679299 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77a6d4-4172-4d2c-80b5-bb764e9fff72 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:00.679811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77a6d4-4172-4d2c-80b5-bb764e9fff72 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:00.680323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77a6d4-4172-4d2c-80b5-bb764e9fff72 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:00.684861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77a6d4-4172-4d2c-80b5-bb764e9fff72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:00.684861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77a6d4-4172-4d2c-80b5-bb764e9fff72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:00.686757 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77a6d4-4172-4d2c-80b5-bb764e9fff72 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:00.706689 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66887b8e-31d2-4a3a-8c4b-4782832a9167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:00.707424 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66887b8e-31d2-4a3a-8c4b-4782832a9167 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:00.707685 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66887b8e-31d2-4a3a-8c4b-4782832a9167 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:00.707966 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66887b8e-31d2-4a3a-8c4b-4782832a9167 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:00.708693 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66887b8e-31d2-4a3a-8c4b-4782832a9167 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:00.708998 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66887b8e-31d2-4a3a-8c4b-4782832a9167 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:00.717431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 340/676] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:00.717431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77a6d4-4172-4d2c-80b5-bb764e9fff72 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:00.717812 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66887b8e-31d2-4a3a-8c4b-4782832a9167 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:00.744661 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 337/677] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:00.747405 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66887b8e-31d2-4a3a-8c4b-4782832a9167 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:00.812254 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:00.858828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77a6d4-4172-4d2c-80b5-bb764e9fff72 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:00.919039 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a212b5d-ea31-4c6b-b619-f6d21ee7d725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:00.920306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a212b5d-ea31-4c6b-b619-f6d21ee7d725 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:00.920562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a212b5d-ea31-4c6b-b619-f6d21ee7d725 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:00.921038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a212b5d-ea31-4c6b-b619-f6d21ee7d725 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:00.922579 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a212b5d-ea31-4c6b-b619-f6d21ee7d725 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:00.923127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a212b5d-ea31-4c6b-b619-f6d21ee7d725 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:00.926040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a212b5d-ea31-4c6b-b619-f6d21ee7d725 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:00.952659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66887b8e-31d2-4a3a-8c4b-4782832a9167 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:00.963015 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 341/678] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:17:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:00.965124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a212b5d-ea31-4c6b-b619-f6d21ee7d725 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:00.992061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f8d6c-d1ff-4b93-aa1f-6a201a22c365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:00.993328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f8d6c-d1ff-4b93-aa1f-6a201a22c365 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:00.994165 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f8d6c-d1ff-4b93-aa1f-6a201a22c365 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:00.994685 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f8d6c-d1ff-4b93-aa1f-6a201a22c365 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:01.003551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f8d6c-d1ff-4b93-aa1f-6a201a22c365 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:01.005394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f8d6c-d1ff-4b93-aa1f-6a201a22c365 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:01.022651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-129f8d6c-d1ff-4b93-aa1f-6a201a22c365 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:01.022651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:01.022651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:01.022651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:01.022651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:01.022651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:01.022651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:01.022651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:01.022651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:01.022651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:01.022651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:01.022651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:01.022651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:01.022651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:01.022651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:01.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:01.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:01.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:01.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:01.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:01.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:01.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:01.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:01.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:01.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:01.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:01.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:01.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:01.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:01.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:01.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:01.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:01.024677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:01.024677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:01.024677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:01.024677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:01.024677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:01.024677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:01.024677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:01.024677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:01.027060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:01.029025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 338/679] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:01.036344 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f8d6c-d1ff-4b93-aa1f-6a201a22c365 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:01.140065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:01.146508 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a212b5d-ea31-4c6b-b619-f6d21ee7d725 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:01.167748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f8d6c-d1ff-4b93-aa1f-6a201a22c365 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:01.189068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd220c99-4c2c-4b5c-bffd-eb930571b5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:01.190756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd220c99-4c2c-4b5c-bffd-eb930571b5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:01.191234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd220c99-4c2c-4b5c-bffd-eb930571b5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:01.191720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd220c99-4c2c-4b5c-bffd-eb930571b5a5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:01.205966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd220c99-4c2c-4b5c-bffd-eb930571b5a5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:01.208631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd220c99-4c2c-4b5c-bffd-eb930571b5a5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:01.215095 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487c5bc-9b4f-4769-91d1-980e8cd2e728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:01.215808 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487c5bc-9b4f-4769-91d1-980e8cd2e728 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:01.216089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487c5bc-9b4f-4769-91d1-980e8cd2e728 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:01.216363 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487c5bc-9b4f-4769-91d1-980e8cd2e728 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:01.217192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487c5bc-9b4f-4769-91d1-980e8cd2e728 None admin] RBAC: Adding query filter params (role.id=5cc9ca1bc48b4d749d21d7111b14e187, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=c92d7ec81ddc4da7a33f9dbb756e59b9) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:01.217533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487c5bc-9b4f-4769-91d1-980e8cd2e728 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:01.219034 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487c5bc-9b4f-4769-91d1-980e8cd2e728 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:01.233330 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 339/680] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:17:01 2022] GET /identity/v3/role_assignments?user.id=c92d7ec81ddc4da7a33f9dbb756e59b9&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=5cc9ca1bc48b4d749d21d7111b14e187 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:01.233330 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487c5bc-9b4f-4769-91d1-980e8cd2e728 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:01.236395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-fd220c99-4c2c-4b5c-bffd-eb930571b5a5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 06:17:01.236395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:01.236395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:01.236395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:01.236395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:01.236395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:01.236395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:01.236395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:01.236395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:01.236395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:01.236395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:01.236395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:01.236395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:17:01.236395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:17:01.236395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:17:01.236395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:01.237691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:01.238755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:01.238755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:01.238755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:17:01.238755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:17:01.238755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:17:01.238755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:17:01.238755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:17:01.238755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:17:01.238755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 06:17:01.238755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:01.238755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 342/681] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:17:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:01.240298 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e741982-f94a-43c9-8350-5ad04a6765c2 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:01.240731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e741982-f94a-43c9-8350-5ad04a6765c2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:01.241150 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e741982-f94a-43c9-8350-5ad04a6765c2 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:01.243743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 343/682] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:01.246000 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e741982-f94a-43c9-8350-5ad04a6765c2 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:01.422778 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487c5bc-9b4f-4769-91d1-980e8cd2e728 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:01.486254 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:01.497119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f5a98-8f6f-47a4-a79b-faf0019cad6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:01.497885 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f5a98-8f6f-47a4-a79b-faf0019cad6c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:01.498171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f5a98-8f6f-47a4-a79b-faf0019cad6c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:01.498449 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f5a98-8f6f-47a4-a79b-faf0019cad6c None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:01.499250 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f5a98-8f6f-47a4-a79b-faf0019cad6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:01.500882 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f5a98-8f6f-47a4-a79b-faf0019cad6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:01.502595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f5a98-8f6f-47a4-a79b-faf0019cad6c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:01.516050 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e741982-f94a-43c9-8350-5ad04a6765c2 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:01.517057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 340/683] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:17:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:01.518269 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6ce87-13db-465f-8d86-ca5928d10a6a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:01.518512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6ce87-13db-465f-8d86-ca5928d10a6a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:01.518792 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6ce87-13db-465f-8d86-ca5928d10a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:01.541383 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-81b6ce87-13db-465f-8d86-ca5928d10a6a None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:01.543179 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-81b6ce87-13db-465f-8d86-ca5928d10a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:01.545732 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:01.595803 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0051b57-02b3-41ef-9309-ee1e84fb127f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:01.597579 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0051b57-02b3-41ef-9309-ee1e84fb127f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:01.598364 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0051b57-02b3-41ef-9309-ee1e84fb127f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:01.599152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0051b57-02b3-41ef-9309-ee1e84fb127f None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:01.601086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0051b57-02b3-41ef-9309-ee1e84fb127f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:01.602367 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0051b57-02b3-41ef-9309-ee1e84fb127f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:01.607063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0051b57-02b3-41ef-9309-ee1e84fb127f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:01.653638 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 344/684] 10.222.0.65 () {56 vars in 1135 bytes} [Thu May 19 06:17:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:01.656037 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0051b57-02b3-41ef-9309-ee1e84fb127f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:01.677331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 341/685] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:17:01.677331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6ce87-13db-465f-8d86-ca5928d10a6a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:01.887690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6ce87-13db-465f-8d86-ca5928d10a6a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:01.912262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0051b57-02b3-41ef-9309-ee1e84fb127f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:01.951052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402a199-55eb-4eee-9e09-07f625c182f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:01.951808 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402a199-55eb-4eee-9e09-07f625c182f5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:01.952114 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402a199-55eb-4eee-9e09-07f625c182f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:01.952399 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402a199-55eb-4eee-9e09-07f625c182f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:01.963863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:01.993864 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31490c05-1784-4724-b2d9-5c30f87c25c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:01.996687 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31490c05-1784-4724-b2d9-5c30f87c25c4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:01.997727 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31490c05-1784-4724-b2d9-5c30f87c25c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:01.998507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31490c05-1784-4724-b2d9-5c30f87c25c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:02.000551 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31490c05-1784-4724-b2d9-5c30f87c25c4 None admin] RBAC: Adding query filter params (role.id=5cc9ca1bc48b4d749d21d7111b14e187, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=226f4700a516415cbbd272fdfad7af58) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:02.001516 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31490c05-1784-4724-b2d9-5c30f87c25c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:02.004563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31490c05-1784-4724-b2d9-5c30f87c25c4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:02.019552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 345/686] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:17:01 2022] GET /identity/v3/role_assignments?user.id=226f4700a516415cbbd272fdfad7af58&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=5cc9ca1bc48b4d749d21d7111b14e187 => generated 608 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:02.021870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31490c05-1784-4724-b2d9-5c30f87c25c4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:02.054994 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f402a199-55eb-4eee-9e09-07f625c182f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:02.056512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f402a199-55eb-4eee-9e09-07f625c182f5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:02.077683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:02.196864 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 342/687] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:17:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:17:02.216093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402a199-55eb-4eee-9e09-07f625c182f5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:02.363773 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:02.390130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31490c05-1784-4724-b2d9-5c30f87c25c4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:02.431175 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0ed4e-7617-4f42-b7b2-500e440adc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:02.431990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0ed4e-7617-4f42-b7b2-500e440adc4d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:02.432274 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0ed4e-7617-4f42-b7b2-500e440adc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:02.432554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0ed4e-7617-4f42-b7b2-500e440adc4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:02.445961 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0ed4e-7617-4f42-b7b2-500e440adc4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:17:02.460835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-e1d0ed4e-7617-4f42-b7b2-500e440adc4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:17:02.460835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:02.460835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:02.460835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:02.460835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:02.460835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:02.460835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:02.460835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:02.460835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:02.460835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:02.460835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:02.460835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:02.460835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:17:02.460835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:17:02.460835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:17:02.460835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:17:02.461856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:17:02.461856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:17:02.461856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:17:02.461856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:17:02.461856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:02.461856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:02.461856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:02.461856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:02.461856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:02.461856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:02.461856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:02.461856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:02.461856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:02.461856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:02.461856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:02.461856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:02.462723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:02.462723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:02.462723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:02.462723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:02.462723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:17:02.462723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:17:02.462723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:17:02.462723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:17:02.462723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:17:02.462723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:17:02.462723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:17:02.462723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:02.464542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 346/688] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:17:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 443 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:02.474247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0ed4e-7617-4f42-b7b2-500e440adc4d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:02.485601 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402a199-55eb-4eee-9e09-07f625c182f5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:02.536772 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829e101-1310-4eed-aaa7-03e9ff54c13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:02.537639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829e101-1310-4eed-aaa7-03e9ff54c13b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:02.537928 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829e101-1310-4eed-aaa7-03e9ff54c13b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:02.538209 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829e101-1310-4eed-aaa7-03e9ff54c13b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:02.563714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829e101-1310-4eed-aaa7-03e9ff54c13b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:02.566710 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829e101-1310-4eed-aaa7-03e9ff54c13b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:02.586854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-c829e101-1310-4eed-aaa7-03e9ff54c13b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:02.586854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:02.586854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:02.586854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:02.586854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:02.586854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:02.586854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:02.586854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:02.586854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:02.586854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:02.586854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:02.586854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:02.586854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:02.586854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:02.586854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:02.588515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:02.588515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:02.588515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:02.588515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:02.588515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:02.588515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:02.588515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:02.588515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:02.588515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:02.588515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:02.588515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:02.588515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:02.588515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:02.588515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:02.588515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:02.588515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:02.588515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:02.591536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:02.591536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:02.591536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:02.591536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:02.591536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 343/689] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:02.595924 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829e101-1310-4eed-aaa7-03e9ff54c13b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:02.618356 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:02.687721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0ed4e-7617-4f42-b7b2-500e440adc4d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:02.749921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6805e6e-4d02-4e59-9fba-d3f168222c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:02.751222 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6805e6e-4d02-4e59-9fba-d3f168222c94 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:02.752321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6805e6e-4d02-4e59-9fba-d3f168222c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:02.753042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6805e6e-4d02-4e59-9fba-d3f168222c94 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:02.754278 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6805e6e-4d02-4e59-9fba-d3f168222c94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:02.755477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6805e6e-4d02-4e59-9fba-d3f168222c94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:02.757307 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6805e6e-4d02-4e59-9fba-d3f168222c94 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:02.779996 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 347/690] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:17:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:02.786038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6805e6e-4d02-4e59-9fba-d3f168222c94 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:02.819927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:02.843493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829e101-1310-4eed-aaa7-03e9ff54c13b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:02.938553 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e35e48-c98c-4e2f-a4b2-4b656b6ca13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:02.940476 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e35e48-c98c-4e2f-a4b2-4b656b6ca13d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:02.944545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e35e48-c98c-4e2f-a4b2-4b656b6ca13d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:02.945572 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e35e48-c98c-4e2f-a4b2-4b656b6ca13d None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:02.948199 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e35e48-c98c-4e2f-a4b2-4b656b6ca13d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:02.952574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e35e48-c98c-4e2f-a4b2-4b656b6ca13d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:02.955635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e35e48-c98c-4e2f-a4b2-4b656b6ca13d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:02.989509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 344/691] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:02.993853 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e35e48-c98c-4e2f-a4b2-4b656b6ca13d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:03.041258 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6805e6e-4d02-4e59-9fba-d3f168222c94 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:03.111404 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4d533-867d-44a0-85c0-3ba3461de2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:03.112758 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4d533-867d-44a0-85c0-3ba3461de2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:03.113253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4d533-867d-44a0-85c0-3ba3461de2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:03.115004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4d533-867d-44a0-85c0-3ba3461de2f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:03.128025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:03.148958 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4d533-867d-44a0-85c0-3ba3461de2f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:03.150508 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4d533-867d-44a0-85c0-3ba3461de2f7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:03.178077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-3fe4d533-867d-44a0-85c0-3ba3461de2f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:03.178077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:03.178077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:03.178077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:03.178077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:03.178077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:03.178077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:03.178077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:03.178077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:03.178077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:03.178077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:03.178077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:03.178077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:03.178077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:03.178077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:03.179126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:03.179126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:03.179126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:03.179126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:03.179126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:03.179126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:03.179126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:03.179126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:03.179126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:03.179126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:03.179126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:03.179126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:03.179126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:03.179126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:03.179126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:03.179126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:03.179126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:03.180021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:03.180021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:03.180021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:03.180021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:03.187249 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 348/692] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:03.190527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4d533-867d-44a0-85c0-3ba3461de2f7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:03.245861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e35e48-c98c-4e2f-a4b2-4b656b6ca13d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:03.284003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33347d-36a9-4631-b476-4b890f6e4416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:03.284910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33347d-36a9-4631-b476-4b890f6e4416 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:03.285266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33347d-36a9-4631-b476-4b890f6e4416 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:03.285638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33347d-36a9-4631-b476-4b890f6e4416 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:03.293563 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33347d-36a9-4631-b476-4b890f6e4416 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:03.295061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33347d-36a9-4631-b476-4b890f6e4416 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:03.312733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-ee33347d-36a9-4631-b476-4b890f6e4416 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:03.312733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:03.312733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:03.312733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:03.312733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:03.312733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:03.312733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:03.312733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:03.312733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:03.312733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:03.312733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:03.312733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:03.312733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:03.312733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:03.312733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:03.314077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:03.314077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:03.314077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:03.314077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:03.314077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:03.314077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:03.314077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:03.314077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:03.314077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:03.314077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:03.314077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:03.314077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:03.314077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:03.314077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:03.314077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:03.314077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:03.314077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:03.315262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:03.315262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:03.315262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:03.315262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:03.315262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:03.315262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:03.315262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:03.315262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:03.315262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 345/693] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:03.316203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b156aa-193d-4e7b-a513-c585a8827c8b None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:03.316203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b156aa-193d-4e7b-a513-c585a8827c8b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:03.316203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b156aa-193d-4e7b-a513-c585a8827c8b None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:03.322368 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 346/694] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:03.329436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b156aa-193d-4e7b-a513-c585a8827c8b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:03.342445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4d533-867d-44a0-85c0-3ba3461de2f7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:03.346877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:03.408041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e7ff1-36cd-4267-9fe0-6c1d0bf24cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:03.409339 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e7ff1-36cd-4267-9fe0-6c1d0bf24cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:03.409932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e7ff1-36cd-4267-9fe0-6c1d0bf24cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:03.410516 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e7ff1-36cd-4267-9fe0-6c1d0bf24cd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:03.411925 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e7ff1-36cd-4267-9fe0-6c1d0bf24cd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:03.412568 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e7ff1-36cd-4267-9fe0-6c1d0bf24cd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:03.415202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e7ff1-36cd-4267-9fe0-6c1d0bf24cd9 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:03.443956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 349/695] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:03.443956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b90136-c9bd-4137-99a1-14fa77a2c20e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:03.443956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b90136-c9bd-4137-99a1-14fa77a2c20e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:03.443956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b90136-c9bd-4137-99a1-14fa77a2c20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:03.472065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-51b90136-c9bd-4137-99a1-14fa77a2c20e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:03.473686 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-51b90136-c9bd-4137-99a1-14fa77a2c20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:03.535542 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b156aa-193d-4e7b-a513-c585a8827c8b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:03.552585 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:03.605633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 350/696] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:17:03.606338 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bde3c1-55f0-4815-b373-94b40471b275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:03.607361 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b90136-c9bd-4137-99a1-14fa77a2c20e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:03.607983 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bde3c1-55f0-4815-b373-94b40471b275 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:03.608683 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bde3c1-55f0-4815-b373-94b40471b275 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:03.609435 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bde3c1-55f0-4815-b373-94b40471b275 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:03.611131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bde3c1-55f0-4815-b373-94b40471b275 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:03.612033 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bde3c1-55f0-4815-b373-94b40471b275 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:03.622580 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bde3c1-55f0-4815-b373-94b40471b275 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:03.660446 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 347/697] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:17:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:03.666778 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bde3c1-55f0-4815-b373-94b40471b275 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:03.794240 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b90136-c9bd-4137-99a1-14fa77a2c20e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:03.817715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:03.861275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98cba6b-1b5a-48a9-8742-715ca00c20f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:03.862705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cba6b-1b5a-48a9-8742-715ca00c20f2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:03.863282 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cba6b-1b5a-48a9-8742-715ca00c20f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:03.863832 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cba6b-1b5a-48a9-8742-715ca00c20f2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:03.877332 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98cba6b-1b5a-48a9-8742-715ca00c20f2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:03.879943 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98cba6b-1b5a-48a9-8742-715ca00c20f2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:03.892703 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bde3c1-55f0-4815-b373-94b40471b275 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:03.904073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-b98cba6b-1b5a-48a9-8742-715ca00c20f2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 06:17:03.904073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:03.904073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:03.904073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:03.904073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:03.904073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:03.904073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:03.904073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:03.904073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:03.904073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:03.904073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:03.904073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:03.904073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:17:03.904073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:17:03.904073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:17:03.904073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:03.905563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:03.906633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:03.906633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:03.906633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:17:03.906633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:17:03.906633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:17:03.906633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:17:03.906633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:17:03.906633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:17:03.906633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 06:17:03.906633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:03.913387 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 351/698] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 06:17:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:03.915683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98cba6b-1b5a-48a9-8742-715ca00c20f2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:03.950440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9a171-6821-49a0-a696-e4b348800075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:03.951544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9a171-6821-49a0-a696-e4b348800075 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:03.952029 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9a171-6821-49a0-a696-e4b348800075 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:03.952484 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9a171-6821-49a0-a696-e4b348800075 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:03.954309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9a171-6821-49a0-a696-e4b348800075 None admin] RBAC: Adding query filter params (role.id=5cc9ca1bc48b4d749d21d7111b14e187, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=5fc205d053d54506bb6c9227e4a1b044) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:03.956550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9a171-6821-49a0-a696-e4b348800075 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:03.963266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9a171-6821-49a0-a696-e4b348800075 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:03.973236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 348/699] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:17:03 2022] GET /identity/v3/role_assignments?user.id=5fc205d053d54506bb6c9227e4a1b044&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=5cc9ca1bc48b4d749d21d7111b14e187 => generated 608 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:03.974845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9a171-6821-49a0-a696-e4b348800075 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:04.002645 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:04.177110 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98cba6b-1b5a-48a9-8742-715ca00c20f2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:04.234021 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9a171-6821-49a0-a696-e4b348800075 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:04.235053 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885f577-b4a7-4bc4-b1d5-2451789e354b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:04.236074 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885f577-b4a7-4bc4-b1d5-2451789e354b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:04.236195 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885f577-b4a7-4bc4-b1d5-2451789e354b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:04.236418 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885f577-b4a7-4bc4-b1d5-2451789e354b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:04.249815 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885f577-b4a7-4bc4-b1d5-2451789e354b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:04.253049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885f577-b4a7-4bc4-b1d5-2451789e354b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:04.278860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-1885f577-b4a7-4bc4-b1d5-2451789e354b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:04.278860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:04.278860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:04.278860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:04.278860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:04.278860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:04.278860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:04.278860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:04.278860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:04.278860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:04.278860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:04.278860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:04.278860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:04.278860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:04.278860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:04.280897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:04.280897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:04.280897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:04.280897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:04.280897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:04.280897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:04.280897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:04.280897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:04.280897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:04.280897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:04.280897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:04.280897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:04.280897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:04.280897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:04.280897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:04.280897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:04.280897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:04.282764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:04.282764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:04.282764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:04.282764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:04.282764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 352/700] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:04.282764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046d18e-e193-4ada-9de2-e8c73cd606ec None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:04.282764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046d18e-e193-4ada-9de2-e8c73cd606ec None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:04.282764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046d18e-e193-4ada-9de2-e8c73cd606ec None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:04.284676 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 353/701] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:04.289647 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf45eb2-ff06-4842-9e15-685ea92124b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:04.290263 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046d18e-e193-4ada-9de2-e8c73cd606ec None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:04.290617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf45eb2-ff06-4842-9e15-685ea92124b7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:04.291118 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf45eb2-ff06-4842-9e15-685ea92124b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:04.291628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf45eb2-ff06-4842-9e15-685ea92124b7 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:04.293999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf45eb2-ff06-4842-9e15-685ea92124b7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:04.294562 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf45eb2-ff06-4842-9e15-685ea92124b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:04.296945 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf45eb2-ff06-4842-9e15-685ea92124b7 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:04.301175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:04.342714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 349/702] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 06:17:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:04.345320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59473e-5e67-41fe-a771-b5e8eb780b15 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:04.345612 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59473e-5e67-41fe-a771-b5e8eb780b15 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:04.345919 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59473e-5e67-41fe-a771-b5e8eb780b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:04.372317 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-7a59473e-5e67-41fe-a771-b5e8eb780b15 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:04.374016 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7a59473e-5e67-41fe-a771-b5e8eb780b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:04.460475 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 350/703] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:17:04.461255 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59473e-5e67-41fe-a771-b5e8eb780b15 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:04.491395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:04.496559 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046d18e-e193-4ada-9de2-e8c73cd606ec None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:04.551962 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc825631-5a7b-4a45-a613-2916f8817fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:04.552676 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc825631-5a7b-4a45-a613-2916f8817fcd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:04.552961 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc825631-5a7b-4a45-a613-2916f8817fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:04.553240 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc825631-5a7b-4a45-a613-2916f8817fcd None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:04.554030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc825631-5a7b-4a45-a613-2916f8817fcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:04.554342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc825631-5a7b-4a45-a613-2916f8817fcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:04.555754 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc825631-5a7b-4a45-a613-2916f8817fcd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:04.572956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 354/704] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:04.574795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc825631-5a7b-4a45-a613-2916f8817fcd None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:04.694666 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59473e-5e67-41fe-a771-b5e8eb780b15 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:04.743452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:04.770811 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe6fdd-9547-49c3-a7d7-719aad6db6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:04.771776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe6fdd-9547-49c3-a7d7-719aad6db6ce None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:04.772234 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe6fdd-9547-49c3-a7d7-719aad6db6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:04.772705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe6fdd-9547-49c3-a7d7-719aad6db6ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:04.777097 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc825631-5a7b-4a45-a613-2916f8817fcd None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:04.847880 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af9fa8-fbb9-40b9-bbb2-3a69a3e1b01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:04.850713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af9fa8-fbb9-40b9-bbb2-3a69a3e1b01e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:04.851555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af9fa8-fbb9-40b9-bbb2-3a69a3e1b01e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:04.852341 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af9fa8-fbb9-40b9-bbb2-3a69a3e1b01e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:04.853943 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af9fa8-fbb9-40b9-bbb2-3a69a3e1b01e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:04.856947 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af9fa8-fbb9-40b9-bbb2-3a69a3e1b01e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:04.886073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-38af9fa8-fbb9-40b9-bbb2-3a69a3e1b01e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 06:17:04.886073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:04.886073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:04.886073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:04.886073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:04.886073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:04.886073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:04.886073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:04.886073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 06:17:04.886073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:04.886073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:04.886073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:04.886073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:04.886073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:04.886073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:04.886073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:04.887461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:04.887461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:04.887461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:04.887461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:04.887461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:04.887461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:04.887461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:04.887461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:04.887461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:04.887461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:04.887461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:04.887461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:04.887461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:04.887461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:04.887461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:04.887461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:04.889127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:04.889127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:04.889127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:04.889127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:04.889127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:04.889127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:04.889127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:04.889127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:04.889127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:04.889127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:04.889127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:04.889127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:04.889127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:04.889127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 06:17:04.889127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:04.894626 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 355/705] 10.222.0.65 () {56 vars in 1129 bytes} [Thu May 19 06:17:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:04.896522 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af9fa8-fbb9-40b9-bbb2-3a69a3e1b01e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:04.928440 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:04.932315 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe6fdd-9547-49c3-a7d7-719aad6db6ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:04.936478 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe6fdd-9547-49c3-a7d7-719aad6db6ce None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:05.090259 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af9fa8-fbb9-40b9-bbb2-3a69a3e1b01e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:05.102278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 351/706] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:17:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:17:05.105147 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe6fdd-9547-49c3-a7d7-719aad6db6ce None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:05.134995 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639336df-efe3-43f5-bbf1-ec088e9dad9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:05.136421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:05.136903 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639336df-efe3-43f5-bbf1-ec088e9dad9f None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:05.137193 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639336df-efe3-43f5-bbf1-ec088e9dad9f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:05.138363 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639336df-efe3-43f5-bbf1-ec088e9dad9f None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:05.138830 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639336df-efe3-43f5-bbf1-ec088e9dad9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:05.140441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639336df-efe3-43f5-bbf1-ec088e9dad9f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:05.163434 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 356/707] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:17:04 2022] POST /identity/v3/users => generated 254 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:17:05.164870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd828d6f-e0f7-4658-980b-1353331f8f94 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:05.165160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd828d6f-e0f7-4658-980b-1353331f8f94 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:05.165486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd828d6f-e0f7-4658-980b-1353331f8f94 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:05.172523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 357/708] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:05.173542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd828d6f-e0f7-4658-980b-1353331f8f94 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:05.266004 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe6fdd-9547-49c3-a7d7-719aad6db6ce None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:05.304526 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:05.341020 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab59bef-7624-4be6-bcdc-e62c9838fcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:05.342817 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab59bef-7624-4be6-bcdc-e62c9838fcd7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:05.344011 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab59bef-7624-4be6-bcdc-e62c9838fcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:05.344958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab59bef-7624-4be6-bcdc-e62c9838fcd7 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:05.346678 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab59bef-7624-4be6-bcdc-e62c9838fcd7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:05.347670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab59bef-7624-4be6-bcdc-e62c9838fcd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:05.350827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab59bef-7624-4be6-bcdc-e62c9838fcd7 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:05.359473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd828d6f-e0f7-4658-980b-1353331f8f94 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:05.361406 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 352/709] 10.222.0.65 () {56 vars in 1124 bytes} [Thu May 19 06:17:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:05.364043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9b523-be0c-4e20-9ddb-d505cba1af91 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:05.364914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9b523-be0c-4e20-9ddb-d505cba1af91 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:05.365873 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9b523-be0c-4e20-9ddb-d505cba1af91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:05.399734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-92c9b523-be0c-4e20-9ddb-d505cba1af91 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:05.400997 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-92c9b523-be0c-4e20-9ddb-d505cba1af91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:05.412336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172cf7c-46b0-49a9-91d6-bf4cd54203eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:05.414273 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172cf7c-46b0-49a9-91d6-bf4cd54203eb None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:05.414997 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172cf7c-46b0-49a9-91d6-bf4cd54203eb None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:05.415723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172cf7c-46b0-49a9-91d6-bf4cd54203eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:05.432162 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172cf7c-46b0-49a9-91d6-bf4cd54203eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:05.435459 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172cf7c-46b0-49a9-91d6-bf4cd54203eb None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:05.459149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-3172cf7c-46b0-49a9-91d6-bf4cd54203eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:05.459149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:05.459149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:05.459149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:05.459149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:05.459149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:05.459149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:05.459149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:05.459149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:05.459149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:05.459149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:05.459149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:05.459149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:05.459149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:05.459149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:05.460117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:05.460117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:05.460117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:05.460117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:05.460117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:05.460117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:05.460117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:05.460117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:05.460117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:05.460117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:05.460117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:05.460117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:05.460117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:05.460117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:05.460117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:05.460117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:05.460117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:05.461086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:05.461086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:05.461086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:05.461086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:05.461086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 358/710] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:05.461407 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172cf7c-46b0-49a9-91d6-bf4cd54203eb None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:05.498245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 353/711] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:17:05.499376 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9b523-be0c-4e20-9ddb-d505cba1af91 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:05.550576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:05.652939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9b523-be0c-4e20-9ddb-d505cba1af91 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:05.655442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172cf7c-46b0-49a9-91d6-bf4cd54203eb None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:05.692603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ca1ce-43ed-4405-8298-7fcd9a999a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:05.693329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ca1ce-43ed-4405-8298-7fcd9a999a91 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:05.693618 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ca1ce-43ed-4405-8298-7fcd9a999a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:05.693900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ca1ce-43ed-4405-8298-7fcd9a999a91 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:05.694716 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ca1ce-43ed-4405-8298-7fcd9a999a91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:05.695579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ca1ce-43ed-4405-8298-7fcd9a999a91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:05.696498 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ca1ce-43ed-4405-8298-7fcd9a999a91 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:05.701531 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:05.711623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 354/712] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:05.712598 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ca1ce-43ed-4405-8298-7fcd9a999a91 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:05.733398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c05abb8-06c5-4dd5-83ca-5df00c5286f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:05.735037 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05abb8-06c5-4dd5-83ca-5df00c5286f0 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:05.735964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05abb8-06c5-4dd5-83ca-5df00c5286f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:05.736904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05abb8-06c5-4dd5-83ca-5df00c5286f0 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:05.738625 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c05abb8-06c5-4dd5-83ca-5df00c5286f0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:05.739477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c05abb8-06c5-4dd5-83ca-5df00c5286f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:05.742401 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c05abb8-06c5-4dd5-83ca-5df00c5286f0 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:05.755360 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 359/713] 10.222.0.65 () {56 vars in 1124 bytes} [Thu May 19 06:17:05 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:05.757420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c05abb8-06c5-4dd5-83ca-5df00c5286f0 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:05.817116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:05.847975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ca1ce-43ed-4405-8298-7fcd9a999a91 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:05.893425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d64d09-2e69-4dd0-982b-caee4a609f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:05.894248 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d64d09-2e69-4dd0-982b-caee4a609f5e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:05.894543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d64d09-2e69-4dd0-982b-caee4a609f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:05.894806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d64d09-2e69-4dd0-982b-caee4a609f5e None admin] PATH_INFO: `/v3/services/network` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:05.895477 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d64d09-2e69-4dd0-982b-caee4a609f5e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:05.897146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d64d09-2e69-4dd0-982b-caee4a609f5e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:05.914340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-d3d64d09-2e69-4dd0-982b-caee4a609f5e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 06:17:05.914340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:05.914340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:05.914340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:05.914340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:05.914340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:05.914340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:05.914340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:05.914340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 06:17:05.914340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:05.914340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:05.914340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:05.914340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:05.914340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:05.914340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:05.914340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:05.914340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:05.916486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:05.916486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:05.916486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:05.916486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:05.916486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:05.916486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:05.916486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:05.916486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:05.916486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:05.916486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:05.916486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:05.916486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:05.916486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:05.916486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:05.916486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:05.916486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:05.918611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:05.918611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:05.918611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:05.918611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:05.918611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:05.918611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:05.918611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:05.918611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:05.918611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:05.918611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:05.918611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:05.918611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:05.918611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 06:17:05.918611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:05.921850 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 355/714] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:05 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:05.924426 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d64d09-2e69-4dd0-982b-caee4a609f5e None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:05.975710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c05abb8-06c5-4dd5-83ca-5df00c5286f0 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:06.064967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8c1f2-6c5f-476d-bffc-9095a87d135e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:06.065818 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8c1f2-6c5f-476d-bffc-9095a87d135e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:06.066100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8c1f2-6c5f-476d-bffc-9095a87d135e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:06.066378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8c1f2-6c5f-476d-bffc-9095a87d135e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:06.086414 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8c1f2-6c5f-476d-bffc-9095a87d135e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:06.087937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8c1f2-6c5f-476d-bffc-9095a87d135e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:06.115202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-0ab8c1f2-6c5f-476d-bffc-9095a87d135e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:06.115202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:06.115202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:06.115202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:06.115202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:06.115202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:06.115202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:06.115202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:06.115202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:06.115202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:06.115202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:06.115202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:06.115202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:06.115202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:06.115202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:06.117022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:06.117022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:06.117022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:06.117022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:06.117022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:06.117022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:06.117022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:06.117022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:06.117022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:06.117022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:06.117022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:06.117022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:06.117022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:06.117022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:06.117022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:06.117022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:06.117022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:06.118850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:06.118850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:06.118850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:06.118850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:06.118850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:06.118850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:06.118850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:06.118850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:06.125094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d64d09-2e69-4dd0-982b-caee4a609f5e None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:06.129518 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 360/715] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:06.130659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8c1f2-6c5f-476d-bffc-9095a87d135e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:06.208319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb4aa9-feaf-4a4c-a2a2-d43caadac436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:06.209484 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb4aa9-feaf-4a4c-a2a2-d43caadac436 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:06.209976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb4aa9-feaf-4a4c-a2a2-d43caadac436 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:06.210453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb4aa9-feaf-4a4c-a2a2-d43caadac436 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:06.217657 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb4aa9-feaf-4a4c-a2a2-d43caadac436 None admin] RBAC: Adding query filter params (name=network) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:06.217657 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb4aa9-feaf-4a4c-a2a2-d43caadac436 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:06.218481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb4aa9-feaf-4a4c-a2a2-d43caadac436 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:06.239801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 356/716] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 06:17:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:06.255655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb4aa9-feaf-4a4c-a2a2-d43caadac436 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:06.285820 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:06.344043 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:06.408366 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8c1f2-6c5f-476d-bffc-9095a87d135e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:06.445519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ba95f-ebe3-4700-9977-d441a8d825a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:06.446306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ba95f-ebe3-4700-9977-d441a8d825a1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:06.446661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ba95f-ebe3-4700-9977-d441a8d825a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:06.447005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ba95f-ebe3-4700-9977-d441a8d825a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:06.447889 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ba95f-ebe3-4700-9977-d441a8d825a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:06.448308 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ba95f-ebe3-4700-9977-d441a8d825a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:06.449916 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ba95f-ebe3-4700-9977-d441a8d825a1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:06.465191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 361/717] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:17:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:06.484784 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ba95f-ebe3-4700-9977-d441a8d825a1 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:06.585267 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb4aa9-feaf-4a4c-a2a2-d43caadac436 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:06.668535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f97406-3731-4104-9dc8-8e355476f6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:06.669235 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f97406-3731-4104-9dc8-8e355476f6cd None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:06.669561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f97406-3731-4104-9dc8-8e355476f6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:06.669837 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f97406-3731-4104-9dc8-8e355476f6cd None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:06.670679 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f97406-3731-4104-9dc8-8e355476f6cd None admin] RBAC: Adding query filter params (type=network) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:06.670996 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f97406-3731-4104-9dc8-8e355476f6cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:06.672415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f97406-3731-4104-9dc8-8e355476f6cd None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:06.688198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 357/718] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 06:17:06 2022] GET /identity/v3/services?type=network => generated 124 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:06.697891 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ba95f-ebe3-4700-9977-d441a8d825a1 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:06.701655 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:06.749975 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e02b5-2196-476c-a823-8c9ef57c16c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:06.750979 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e02b5-2196-476c-a823-8c9ef57c16c1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:06.751449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e02b5-2196-476c-a823-8c9ef57c16c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:06.751932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e02b5-2196-476c-a823-8c9ef57c16c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:06.753000 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e02b5-2196-476c-a823-8c9ef57c16c1 None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=2a09f4878eac4d60a9a7dcedba15c05d) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:06.753543 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e02b5-2196-476c-a823-8c9ef57c16c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:06.755284 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e02b5-2196-476c-a823-8c9ef57c16c1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:06.767339 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 362/719] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:17:06 2022] GET /identity/v3/role_assignments?user.id=2a09f4878eac4d60a9a7dcedba15c05d&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:06.854376 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b2c54-9916-485c-9226-9070ebb9af62 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:06.854949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b2c54-9916-485c-9226-9070ebb9af62 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:06.854949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b2c54-9916-485c-9226-9070ebb9af62 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:06.856877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 358/720] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:06.863156 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5c0c4-94c0-4a3e-9c75-78d1bf4d1852 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:06.863515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5c0c4-94c0-4a3e-9c75-78d1bf4d1852 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:06.863737 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5c0c4-94c0-4a3e-9c75-78d1bf4d1852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:06.890581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-00f5c0c4-94c0-4a3e-9c75-78d1bf4d1852 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:06.891883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-00f5c0c4-94c0-4a3e-9c75-78d1bf4d1852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:06.998170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 363/721] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:17:07.024758 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b2c54-9916-485c-9226-9070ebb9af62 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:07.138902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:07.187138 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b2c54-9916-485c-9226-9070ebb9af62 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:07.237274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda22e0d-aeea-4076-b4df-38a55366ac65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:07.238102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda22e0d-aeea-4076-b4df-38a55366ac65 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:07.238393 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda22e0d-aeea-4076-b4df-38a55366ac65 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:07.238669 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda22e0d-aeea-4076-b4df-38a55366ac65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:07.255026 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda22e0d-aeea-4076-b4df-38a55366ac65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:17:07.270056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-eda22e0d-aeea-4076-b4df-38a55366ac65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:17:07.270056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:07.270056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:07.270056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:07.270056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:07.270056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:07.270056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:07.270056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:07.270056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:07.270056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:07.270056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:07.270056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:07.270056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:17:07.270056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:17:07.270056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:17:07.270056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:17:07.271902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:17:07.271902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:17:07.271902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:17:07.271902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:17:07.271902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:07.271902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:07.271902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:07.271902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:07.271902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:07.271902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:07.271902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:07.271902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:07.271902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:07.271902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:07.271902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:07.271902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:07.273503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:07.273503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:07.273503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:07.273503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:07.273503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:17:07.273503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:17:07.273503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:17:07.273503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:17:07.273503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:17:07.273503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:17:07.273503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:17:07.273503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:07.273503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 359/722] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:17:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:07.278758 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5c0c4-94c0-4a3e-9c75-78d1bf4d1852 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:07.553122 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:07.561634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5c0c4-94c0-4a3e-9c75-78d1bf4d1852 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:07.615279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2b2cd-64dc-4383-af84-f2306eb1c46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:07.615967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2b2cd-64dc-4383-af84-f2306eb1c46c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:07.616249 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2b2cd-64dc-4383-af84-f2306eb1c46c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:07.616528 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2b2cd-64dc-4383-af84-f2306eb1c46c None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:07.617310 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c2b2cd-64dc-4383-af84-f2306eb1c46c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:07.617623 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c2b2cd-64dc-4383-af84-f2306eb1c46c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:07.619014 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c2b2cd-64dc-4383-af84-f2306eb1c46c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:07.634337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 364/723] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:17:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:07.641068 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda22e0d-aeea-4076-b4df-38a55366ac65 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:07.914578 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda22e0d-aeea-4076-b4df-38a55366ac65 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:07.968381 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:07.982690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff74f4f-733f-4579-a812-0a8cad900290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:07.984059 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff74f4f-733f-4579-a812-0a8cad900290 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:07.984570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff74f4f-733f-4579-a812-0a8cad900290 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:07.985065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff74f4f-733f-4579-a812-0a8cad900290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:08.006266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff74f4f-733f-4579-a812-0a8cad900290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:08.009176 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff74f4f-733f-4579-a812-0a8cad900290 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:08.045519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-6ff74f4f-733f-4579-a812-0a8cad900290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:08.045519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:08.045519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:08.045519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:08.045519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:08.045519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:08.045519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:08.045519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:08.045519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:08.045519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:08.045519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:08.045519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:08.045519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:08.045519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:08.045519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:08.046564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:08.046564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:08.046564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:08.046564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:08.046564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:08.046564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:08.046564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:08.046564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:08.046564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:08.046564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:08.046564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:08.046564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:08.046564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:08.046564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:08.046564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:08.046564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:08.046564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:08.047460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:08.047460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:08.047460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:08.047460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:08.052425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 360/724] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:08.054140 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2b2cd-64dc-4383-af84-f2306eb1c46c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:08.237391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2b2cd-64dc-4383-af84-f2306eb1c46c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:08.295427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69dda47-c3cc-43e7-8a89-93751f3f6023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:08.296152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69dda47-c3cc-43e7-8a89-93751f3f6023 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:08.296442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69dda47-c3cc-43e7-8a89-93751f3f6023 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:08.296725 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69dda47-c3cc-43e7-8a89-93751f3f6023 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:08.297512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69dda47-c3cc-43e7-8a89-93751f3f6023 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:08.297821 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69dda47-c3cc-43e7-8a89-93751f3f6023 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:08.299271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69dda47-c3cc-43e7-8a89-93751f3f6023 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:08.320602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 365/725] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:08.330207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff74f4f-733f-4579-a812-0a8cad900290 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:08.627540 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff74f4f-733f-4579-a812-0a8cad900290 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:08.717032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0486f0-89bf-493f-97f3-e61ac0106f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:08.718893 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0486f0-89bf-493f-97f3-e61ac0106f88 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:08.719668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0486f0-89bf-493f-97f3-e61ac0106f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:08.720394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0486f0-89bf-493f-97f3-e61ac0106f88 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:08.740715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0486f0-89bf-493f-97f3-e61ac0106f88 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:08.743951 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0486f0-89bf-493f-97f3-e61ac0106f88 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:08.785931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-ca0486f0-89bf-493f-97f3-e61ac0106f88 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 06:17:08.785931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:08.785931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:08.785931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:08.785931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:08.785931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:08.785931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:08.785931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:08.785931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:08.785931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:08.785931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:08.785931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:08.785931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:17:08.785931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:17:08.785931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:17:08.785931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:08.787730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:08.789623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:08.789623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:08.789623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:17:08.789623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:17:08.789623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:17:08.789623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:17:08.789623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:17:08.789623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:17:08.789623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 06:17:08.789623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:08.794471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 361/726] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:17:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 465 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:08.802441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69dda47-c3cc-43e7-8a89-93751f3f6023 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:08.848426 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:08.987442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69dda47-c3cc-43e7-8a89-93751f3f6023 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:09.039197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a68ec-2c14-4447-a3e5-18fd4b961d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:09.039925 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a68ec-2c14-4447-a3e5-18fd4b961d7e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:09.040215 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a68ec-2c14-4447-a3e5-18fd4b961d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:09.040498 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a68ec-2c14-4447-a3e5-18fd4b961d7e None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:09.041392 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a68ec-2c14-4447-a3e5-18fd4b961d7e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:09.041709 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a68ec-2c14-4447-a3e5-18fd4b961d7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:09.044120 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a68ec-2c14-4447-a3e5-18fd4b961d7e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:09.069969 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 366/727] 10.222.0.65 () {56 vars in 1135 bytes} [Thu May 19 06:17:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:09.077571 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0486f0-89bf-493f-97f3-e61ac0106f88 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:09.202315 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:09.368272 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0486f0-89bf-493f-97f3-e61ac0106f88 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:09.409015 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b061601-060e-4dfb-be65-c4cea2312f8b None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:09.409489 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b061601-060e-4dfb-be65-c4cea2312f8b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:09.409948 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b061601-060e-4dfb-be65-c4cea2312f8b None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:09.412528 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 367/728] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:09.414480 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a0e57-4a80-4ca5-95c5-9654e7080af1 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:09.414946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a0e57-4a80-4ca5-95c5-9654e7080af1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:09.415356 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a0e57-4a80-4ca5-95c5-9654e7080af1 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:09.418109 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 368/729] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:09.428783 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6968ca-2c30-4fab-8fa6-fd47394bc36a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:09.429318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6968ca-2c30-4fab-8fa6-fd47394bc36a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:09.429842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6968ca-2c30-4fab-8fa6-fd47394bc36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:09.449357 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db654599-36ca-4176-b2a6-4d659f27b640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:09.450407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db654599-36ca-4176-b2a6-4d659f27b640 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:09.450942 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db654599-36ca-4176-b2a6-4d659f27b640 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:09.451668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db654599-36ca-4176-b2a6-4d659f27b640 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:09.465976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-fe6968ca-2c30-4fab-8fa6-fd47394bc36a None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:09.468514 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fe6968ca-2c30-4fab-8fa6-fd47394bc36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:09.568063 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db654599-36ca-4176-b2a6-4d659f27b640 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:09.570470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db654599-36ca-4176-b2a6-4d659f27b640 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:09.637462 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 369/730] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 06:17:09.639579 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01fb0f-77ad-4e8c-ba15-0815fe4e4632 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:09.640166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01fb0f-77ad-4e8c-ba15-0815fe4e4632 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:09.640806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01fb0f-77ad-4e8c-ba15-0815fe4e4632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:09.644084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:09.687788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-1e01fb0f-77ad-4e8c-ba15-0815fe4e4632 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:09.689119 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1e01fb0f-77ad-4e8c-ba15-0815fe4e4632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:09.693692 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:09.787900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 362/731] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:17:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 06:17:09.789859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db654599-36ca-4176-b2a6-4d659f27b640 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:09.817613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 370/732] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:17:09.819074 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01fb0f-77ad-4e8c-ba15-0815fe4e4632 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:09.944682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db654599-36ca-4176-b2a6-4d659f27b640 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:09.962245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01fb0f-77ad-4e8c-ba15-0815fe4e4632 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:09.985159 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3277957-6e59-4649-8cdd-28e1d0c493ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:09.985871 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3277957-6e59-4649-8cdd-28e1d0c493ac None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:09.986162 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3277957-6e59-4649-8cdd-28e1d0c493ac None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:09.986446 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3277957-6e59-4649-8cdd-28e1d0c493ac None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:09.987185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3277957-6e59-4649-8cdd-28e1d0c493ac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:09.988611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3277957-6e59-4649-8cdd-28e1d0c493ac None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:10.002176 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:10.017746 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 363/733] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 06:17:09 2022] POST /identity/v3/services => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:17:10.019139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9513f8-d6d5-48fc-8571-2d4431e63507 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:10.019545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9513f8-d6d5-48fc-8571-2d4431e63507 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:10.019705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9513f8-d6d5-48fc-8571-2d4431e63507 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:10.025594 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 364/734] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:10.025594 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9513f8-d6d5-48fc-8571-2d4431e63507 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:10.041865 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccdabfe-5f53-44d3-a233-1e55d43dbd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:10.043299 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccdabfe-5f53-44d3-a233-1e55d43dbd83 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:10.043623 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccdabfe-5f53-44d3-a233-1e55d43dbd83 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:10.044165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccdabfe-5f53-44d3-a233-1e55d43dbd83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:10.051230 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:10.078182 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccdabfe-5f53-44d3-a233-1e55d43dbd83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:10.082619 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccdabfe-5f53-44d3-a233-1e55d43dbd83 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:10.125325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-1ccdabfe-5f53-44d3-a233-1e55d43dbd83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:10.125325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:10.125325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:10.125325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:10.125325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:10.125325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:10.125325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:10.125325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:10.125325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:10.125325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:10.125325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:10.125325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:10.125325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:10.125325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:10.125325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:10.126918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:10.126918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:10.126918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:10.126918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:10.126918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:10.126918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:10.126918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:10.126918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:10.126918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:10.126918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:10.126918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:10.126918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:10.126918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:10.126918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:10.126918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:10.126918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:10.126918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:10.128638 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:10.128638 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:10.128638 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:10.128638 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:10.133000 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 371/735] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:10.137042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba35a93-e36f-4f8b-b6d2-4258237dc492 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:10.138304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba35a93-e36f-4f8b-b6d2-4258237dc492 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:10.139153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba35a93-e36f-4f8b-b6d2-4258237dc492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:10.173576 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-6ba35a93-e36f-4f8b-b6d2-4258237dc492 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:10.176824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6ba35a93-e36f-4f8b-b6d2-4258237dc492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:10.248709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9513f8-d6d5-48fc-8571-2d4431e63507 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:10.312156 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 372/736] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:10 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:17:10.313546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba35a93-e36f-4f8b-b6d2-4258237dc492 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:10.320873 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fad31-88a5-44b7-8d9d-f6d505d8bc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:10.322149 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fad31-88a5-44b7-8d9d-f6d505d8bc52 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:10.325753 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fad31-88a5-44b7-8d9d-f6d505d8bc52 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:10.326362 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fad31-88a5-44b7-8d9d-f6d505d8bc52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:10.344047 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294fad31-88a5-44b7-8d9d-f6d505d8bc52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:17:10.352191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-294fad31-88a5-44b7-8d9d-f6d505d8bc52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:17:10.352191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:10.352191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:10.352191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:10.352191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:10.352191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:10.352191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:10.352191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:10.352191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:10.352191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:10.352191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:10.352191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:10.352191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:17:10.352191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:17:10.352191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:17:10.352191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:17:10.354116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:17:10.354116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:17:10.354116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:17:10.354116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:17:10.354116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:10.354116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:10.354116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:10.354116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:10.354116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:10.354116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:10.354116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:10.354116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:10.354116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:10.354116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:10.354116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:10.354116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:10.355588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:10.355588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:10.355588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:10.355588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:10.355588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:17:10.355588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:17:10.355588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:17:10.355588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:17:10.355588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:17:10.355588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:17:10.355588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:17:10.355588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:10.355588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 365/737] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:17:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:10.357459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab253ac4-e391-4897-bb7d-736eda4c0a11 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:10.357459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab253ac4-e391-4897-bb7d-736eda4c0a11 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:10.357459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab253ac4-e391-4897-bb7d-736eda4c0a11 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:10.358399 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 366/738] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:10.365098 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab253ac4-e391-4897-bb7d-736eda4c0a11 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:10.465420 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:10.486224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba35a93-e36f-4f8b-b6d2-4258237dc492 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:10.507206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:10.532904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178c1e4-db61-4f88-b661-758d402fd2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:10.533780 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178c1e4-db61-4f88-b661-758d402fd2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:10.534067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178c1e4-db61-4f88-b661-758d402fd2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:10.534345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178c1e4-db61-4f88-b661-758d402fd2d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:10.535087 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178c1e4-db61-4f88-b661-758d402fd2d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:10.535394 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178c1e4-db61-4f88-b661-758d402fd2d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:10.536799 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178c1e4-db61-4f88-b661-758d402fd2d7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:10.538231 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab253ac4-e391-4897-bb7d-736eda4c0a11 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:10.554551 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 373/739] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:10.554551 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed7216-b1b0-4299-8582-f0e9478904a5 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:10.554551 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed7216-b1b0-4299-8582-f0e9478904a5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:10.554551 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed7216-b1b0-4299-8582-f0e9478904a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:10.569944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-eeed7216-b1b0-4299-8582-f0e9478904a5 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:10.571171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-eeed7216-b1b0-4299-8582-f0e9478904a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:10.608509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80df5f-3705-4f97-baea-ebfa9e1e67cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:10.609997 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80df5f-3705-4f97-baea-ebfa9e1e67cc None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:10.610430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80df5f-3705-4f97-baea-ebfa9e1e67cc None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:10.610947 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80df5f-3705-4f97-baea-ebfa9e1e67cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:10.632073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80df5f-3705-4f97-baea-ebfa9e1e67cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:17:10.648686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-7e80df5f-3705-4f97-baea-ebfa9e1e67cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:17:10.648686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:10.648686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:10.648686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:10.648686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:10.648686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:10.648686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:10.648686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:10.648686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:10.648686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:10.648686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:10.648686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:10.648686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:17:10.648686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:17:10.648686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:17:10.648686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:17:10.650043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:17:10.650043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:17:10.650043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:17:10.650043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:17:10.650043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:10.650043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:10.650043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:10.650043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:10.650043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:10.650043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:10.650043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:10.650043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:10.650043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:10.650043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:10.650043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:10.650043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:10.650999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:10.650999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:10.650999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:10.650999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:10.650999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:17:10.650999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:17:10.650999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:17:10.650999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:17:10.650999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:17:10.650999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:17:10.650999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:17:10.650999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:10.659986 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 367/740] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:17:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:10.659986 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80df5f-3705-4f97-baea-ebfa9e1e67cc None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:10.661535 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 374/741] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:10 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 06:17:10.663450 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed7216-b1b0-4299-8582-f0e9478904a5 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:10.795250 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80df5f-3705-4f97-baea-ebfa9e1e67cc None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:10.811572 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed7216-b1b0-4299-8582-f0e9478904a5 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:10.820242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:10.836269 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:10.855232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792f9c0-e20e-43c2-96e7-1f883ac8ec9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:10.856056 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792f9c0-e20e-43c2-96e7-1f883ac8ec9d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:10.856338 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792f9c0-e20e-43c2-96e7-1f883ac8ec9d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:10.856617 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792f9c0-e20e-43c2-96e7-1f883ac8ec9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:10.864284 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792f9c0-e20e-43c2-96e7-1f883ac8ec9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:10.864835 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbfaaa-6534-49c1-a4a6-16f2e362f8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:10.865791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792f9c0-e20e-43c2-96e7-1f883ac8ec9d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:10.866487 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbfaaa-6534-49c1-a4a6-16f2e362f8fc None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:10.867233 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbfaaa-6534-49c1-a4a6-16f2e362f8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:10.868105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbfaaa-6534-49c1-a4a6-16f2e362f8fc None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:10.870331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbfaaa-6534-49c1-a4a6-16f2e362f8fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:10.871089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbfaaa-6534-49c1-a4a6-16f2e362f8fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:10.874049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbfaaa-6534-49c1-a4a6-16f2e362f8fc None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:10.881076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-7792f9c0-e20e-43c2-96e7-1f883ac8ec9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:10.881076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:10.881076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:10.881076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:10.881076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:10.881076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:10.881076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:10.881076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:10.881076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:10.881076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:10.881076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:10.881076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:10.881076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:10.881076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:10.881076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:10.882833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:10.882833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:10.882833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:10.882833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:10.882833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:10.882833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:10.882833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:10.882833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:10.882833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:10.882833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:10.882833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:10.882833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:10.882833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:10.882833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:10.882833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:10.882833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:10.882833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:10.884477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:10.884477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:10.884477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:10.884477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:10.884477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:10.884477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:10.884477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:10.884477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:10.884477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 375/742] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:10.884477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792f9c0-e20e-43c2-96e7-1f883ac8ec9d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:10.892510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 368/743] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:17:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:10.894626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbfaaa-6534-49c1-a4a6-16f2e362f8fc None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:11.031283 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792f9c0-e20e-43c2-96e7-1f883ac8ec9d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:11.067133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbfaaa-6534-49c1-a4a6-16f2e362f8fc None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:11.077509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9872ce-6a8d-4db6-8859-f94f306b06b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:11.078396 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9872ce-6a8d-4db6-8859-f94f306b06b5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:11.078933 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9872ce-6a8d-4db6-8859-f94f306b06b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:11.079367 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9872ce-6a8d-4db6-8859-f94f306b06b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:11.080362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9872ce-6a8d-4db6-8859-f94f306b06b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:11.080769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9872ce-6a8d-4db6-8859-f94f306b06b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:11.082228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9872ce-6a8d-4db6-8859-f94f306b06b5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:11.093398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 376/744] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:17:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:11.094279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9872ce-6a8d-4db6-8859-f94f306b06b5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:11.114472 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398c8db-8c12-4965-b8f7-1b3b257b7469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:11.115512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398c8db-8c12-4965-b8f7-1b3b257b7469 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:11.115800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398c8db-8c12-4965-b8f7-1b3b257b7469 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:11.116077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398c8db-8c12-4965-b8f7-1b3b257b7469 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:11.116816 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398c8db-8c12-4965-b8f7-1b3b257b7469 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:11.118289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398c8db-8c12-4965-b8f7-1b3b257b7469 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:11.134254 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 369/745] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 06:17:10 2022] POST /identity/v3/services => generated 240 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:17:11.135101 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398c8db-8c12-4965-b8f7-1b3b257b7469 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:11.171097 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:11.211834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:11.297114 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9872ce-6a8d-4db6-8859-f94f306b06b5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:11.362605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398c8db-8c12-4965-b8f7-1b3b257b7469 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:11.368583 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf498ff-e478-4200-b9af-3c333aa02216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:11.369410 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf498ff-e478-4200-b9af-3c333aa02216 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:11.369702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf498ff-e478-4200-b9af-3c333aa02216 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:11.370152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf498ff-e478-4200-b9af-3c333aa02216 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:11.371129 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf498ff-e478-4200-b9af-3c333aa02216 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:11.371634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf498ff-e478-4200-b9af-3c333aa02216 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:11.374531 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf498ff-e478-4200-b9af-3c333aa02216 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:11.392458 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 377/746] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:17:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:11.395433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf498ff-e478-4200-b9af-3c333aa02216 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:11.445018 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001be05-a583-46c5-8365-d3f1eedfed66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:11.446073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001be05-a583-46c5-8365-d3f1eedfed66 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:11.446789 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001be05-a583-46c5-8365-d3f1eedfed66 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:11.448896 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001be05-a583-46c5-8365-d3f1eedfed66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:11.464355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001be05-a583-46c5-8365-d3f1eedfed66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:11.466329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001be05-a583-46c5-8365-d3f1eedfed66 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:11.502195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-b001be05-a583-46c5-8365-d3f1eedfed66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:11.502195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:11.502195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:11.502195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:11.502195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:11.502195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:11.502195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:11.502195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:11.502195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:11.502195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:11.502195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:11.502195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:11.502195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:11.502195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:11.502195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:11.503736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:11.503736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:11.503736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:11.503736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:11.503736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:11.503736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:11.503736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:11.503736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:11.503736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:11.503736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:11.503736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:11.503736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:11.503736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:11.503736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:11.503736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:11.503736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:11.503736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:11.505189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:11.505189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:11.505189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:11.505189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:11.505189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 370/747] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:11.505596 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001be05-a583-46c5-8365-d3f1eedfed66 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:11.566807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:11.594282 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf498ff-e478-4200-b9af-3c333aa02216 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:11.651716 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194dbdaf-f75f-4ab3-be4e-43873ed16fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:11.652539 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dbdaf-f75f-4ab3-be4e-43873ed16fcb None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:11.652827 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dbdaf-f75f-4ab3-be4e-43873ed16fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:11.653155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dbdaf-f75f-4ab3-be4e-43873ed16fcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:11.672909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:11.679391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194dbdaf-f75f-4ab3-be4e-43873ed16fcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:11.684656 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194dbdaf-f75f-4ab3-be4e-43873ed16fcb None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:11.687548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001be05-a583-46c5-8365-d3f1eedfed66 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:11.723008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-194dbdaf-f75f-4ab3-be4e-43873ed16fcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:11.723008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:11.723008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:11.723008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:11.723008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:11.723008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:11.723008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:11.723008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:11.723008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:11.723008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:11.723008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:11.723008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:11.723008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:11.723008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:11.723008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:11.723991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:11.723991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:11.723991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:11.723991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:11.723991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:11.723991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:11.723991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:11.723991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:11.723991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:11.723991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:11.723991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:11.723991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:11.723991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:11.723991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:11.723991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:11.723991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:11.723991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:11.724896 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:11.724896 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:11.724896 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:11.724896 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:11.727258 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 378/748] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:11.728356 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194dbdaf-f75f-4ab3-be4e-43873ed16fcb None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:11.754242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fdc22-b475-4f32-b123-7ad2b9b76ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:11.755359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fdc22-b475-4f32-b123-7ad2b9b76ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:11.755864 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fdc22-b475-4f32-b123-7ad2b9b76ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:11.756340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fdc22-b475-4f32-b123-7ad2b9b76ad0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:11.757960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fdc22-b475-4f32-b123-7ad2b9b76ad0 None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=5fc205d053d54506bb6c9227e4a1b044) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:11.759487 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fdc22-b475-4f32-b123-7ad2b9b76ad0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:11.764206 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fdc22-b475-4f32-b123-7ad2b9b76ad0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:11.781473 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 371/749] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:17:11 2022] GET /identity/v3/role_assignments?user.id=5fc205d053d54506bb6c9227e4a1b044&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:11.789357 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fdc22-b475-4f32-b123-7ad2b9b76ad0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:12.031824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194dbdaf-f75f-4ab3-be4e-43873ed16fcb None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:12.090087 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ae2aa-feff-41ff-93f5-b5bdbc7cad16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:12.091593 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ae2aa-feff-41ff-93f5-b5bdbc7cad16 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:12.092113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ae2aa-feff-41ff-93f5-b5bdbc7cad16 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:12.092598 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ae2aa-feff-41ff-93f5-b5bdbc7cad16 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:12.093697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ae2aa-feff-41ff-93f5-b5bdbc7cad16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:12.094242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ae2aa-feff-41ff-93f5-b5bdbc7cad16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:12.095860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ae2aa-feff-41ff-93f5-b5bdbc7cad16 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:12.105228 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fdc22-b475-4f32-b123-7ad2b9b76ad0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:12.112688 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:12.116101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 379/750] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:12.123351 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ae2aa-feff-41ff-93f5-b5bdbc7cad16 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:12.184764 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed41a7-5968-4a8c-afc2-ffe4a4533a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:12.185942 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed41a7-5968-4a8c-afc2-ffe4a4533a12 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:12.186445 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed41a7-5968-4a8c-afc2-ffe4a4533a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:12.186987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed41a7-5968-4a8c-afc2-ffe4a4533a12 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:12.188112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed41a7-5968-4a8c-afc2-ffe4a4533a12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:12.188641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed41a7-5968-4a8c-afc2-ffe4a4533a12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:12.190304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed41a7-5968-4a8c-afc2-ffe4a4533a12 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:12.207312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 372/751] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:12.219675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed41a7-5968-4a8c-afc2-ffe4a4533a12 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:12.249145 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:12.376202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ae2aa-feff-41ff-93f5-b5bdbc7cad16 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:12.438889 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55ca60-a083-4bbe-8ae3-cdc170f943e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:12.439787 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55ca60-a083-4bbe-8ae3-cdc170f943e5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:12.440069 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55ca60-a083-4bbe-8ae3-cdc170f943e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:12.440355 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55ca60-a083-4bbe-8ae3-cdc170f943e5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:12.451153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55ca60-a083-4bbe-8ae3-cdc170f943e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:12.452895 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55ca60-a083-4bbe-8ae3-cdc170f943e5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:12.459687 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed41a7-5968-4a8c-afc2-ffe4a4533a12 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:12.472152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-1e55ca60-a083-4bbe-8ae3-cdc170f943e5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 06:17:12.472152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:12.472152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:12.472152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:12.472152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:12.472152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:12.472152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:12.472152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:12.472152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:12.472152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:12.472152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:12.472152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:12.472152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:17:12.472152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:17:12.472152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:17:12.472152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:12.473928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:12.477629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:12.477629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:12.477629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:17:12.477629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:17:12.477629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:17:12.477629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:17:12.477629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:17:12.477629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:17:12.477629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 06:17:12.477629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:12.477629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 380/752] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 06:17:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:12.482529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55ca60-a083-4bbe-8ae3-cdc170f943e5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:12.561347 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:12.606926 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836adc9e-cd52-4fa4-b5f0-75d16a41b1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:12.607652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836adc9e-cd52-4fa4-b5f0-75d16a41b1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:12.607937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836adc9e-cd52-4fa4-b5f0-75d16a41b1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:12.608216 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836adc9e-cd52-4fa4-b5f0-75d16a41b1e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:12.615785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55ca60-a083-4bbe-8ae3-cdc170f943e5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:12.622586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836adc9e-cd52-4fa4-b5f0-75d16a41b1e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:12.624078 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836adc9e-cd52-4fa4-b5f0-75d16a41b1e2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:12.656247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20401352-fb5c-45a2-ade5-3ebc89115b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:12.656930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20401352-fb5c-45a2-ade5-3ebc89115b09 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:12.657263 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20401352-fb5c-45a2-ade5-3ebc89115b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:12.657568 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20401352-fb5c-45a2-ade5-3ebc89115b09 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:12.658425 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20401352-fb5c-45a2-ade5-3ebc89115b09 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:12.658740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20401352-fb5c-45a2-ade5-3ebc89115b09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:12.660245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20401352-fb5c-45a2-ade5-3ebc89115b09 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:12.662190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-836adc9e-cd52-4fa4-b5f0-75d16a41b1e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 06:17:12.662190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:12.662190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:12.662190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:12.662190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:12.662190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:12.662190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:12.662190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:12.662190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:12.662190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:12.662190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:12.662190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:12.662190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:17:12.662190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:17:12.662190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:17:12.662190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:12.664519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:12.673510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:12.673510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:12.673510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:17:12.673510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:17:12.673510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:17:12.673510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:17:12.673510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:17:12.673510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:17:12.673510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 06:17:12.673510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:12.673510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 373/753] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 06:17:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:12.673510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836adc9e-cd52-4fa4-b5f0-75d16a41b1e2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:12.680818 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 381/754] 10.222.0.65 () {56 vars in 1149 bytes} [Thu May 19 06:17:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:12.687346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20401352-fb5c-45a2-ade5-3ebc89115b09 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:12.813577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20401352-fb5c-45a2-ade5-3ebc89115b09 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:12.827425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:12.835519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:12.857825 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3823a27-65b5-4946-87f3-32577081a808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:12.858430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3823a27-65b5-4946-87f3-32577081a808 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:12.858718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3823a27-65b5-4946-87f3-32577081a808 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:12.859000 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3823a27-65b5-4946-87f3-32577081a808 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:12.880555 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836adc9e-cd52-4fa4-b5f0-75d16a41b1e2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:12.926223 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d48c7-5d3d-4ec4-8355-8370515ca618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:12.926718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d48c7-5d3d-4ec4-8355-8370515ca618 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:12.927010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d48c7-5d3d-4ec4-8355-8370515ca618 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:12.927296 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d48c7-5d3d-4ec4-8355-8370515ca618 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:12.928140 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d48c7-5d3d-4ec4-8355-8370515ca618 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:12.928446 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d48c7-5d3d-4ec4-8355-8370515ca618 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:12.930043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d48c7-5d3d-4ec4-8355-8370515ca618 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:12.942644 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3823a27-65b5-4946-87f3-32577081a808 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:12.944495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3823a27-65b5-4946-87f3-32577081a808 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:12.951252 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 374/755] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 06:17:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:12.952689 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b442db-5a7c-40a4-a553-3f7e0abd2243 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:12.952983 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b442db-5a7c-40a4-a553-3f7e0abd2243 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:12.953262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b442db-5a7c-40a4-a553-3f7e0abd2243 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:12.958754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 375/756] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:12.958754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b442db-5a7c-40a4-a553-3f7e0abd2243 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:13.092684 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 382/757] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:17:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:17:13.094873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7e049-280a-4193-aa5a-debfb55c8ddb None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:13.095371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7e049-280a-4193-aa5a-debfb55c8ddb None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:13.095850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7e049-280a-4193-aa5a-debfb55c8ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:13.116839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-14b7e049-280a-4193-aa5a-debfb55c8ddb None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:13.118427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-14b7e049-280a-4193-aa5a-debfb55c8ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:13.134095 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:13.134296 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b442db-5a7c-40a4-a553-3f7e0abd2243 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:13.172515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:13.179312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78179f6e-08fa-4816-a459-ada4c5cf7521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:13.180087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78179f6e-08fa-4816-a459-ada4c5cf7521 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:13.180381 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78179f6e-08fa-4816-a459-ada4c5cf7521 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:13.180659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78179f6e-08fa-4816-a459-ada4c5cf7521 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:13.195335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78179f6e-08fa-4816-a459-ada4c5cf7521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:13.196740 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78179f6e-08fa-4816-a459-ada4c5cf7521 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:13.214025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 383/758] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:13.215702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7e049-280a-4193-aa5a-debfb55c8ddb None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:13.217481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-78179f6e-08fa-4816-a459-ada4c5cf7521 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:13.217481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:13.217481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:13.217481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:13.217481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:13.217481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:13.217481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:13.217481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:13.217481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:13.217481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:13.217481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:13.217481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:13.217481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:13.217481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:13.217481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:13.218409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:13.218409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:13.218409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:13.218409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:13.218409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:13.218409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:13.218409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:13.218409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:13.218409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:13.218409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:13.218409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:13.218409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:13.218409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:13.218409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:13.218409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:13.218409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:13.218409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:13.223993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:13.223993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:13.223993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:13.223993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:13.223993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 376/759] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:13.224457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78179f6e-08fa-4816-a459-ada4c5cf7521 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:13.359266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78179f6e-08fa-4816-a459-ada4c5cf7521 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:13.373826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7e049-280a-4193-aa5a-debfb55c8ddb None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:13.404574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212f16b-e46f-4c8a-8c9b-706a10fb0674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:13.405273 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212f16b-e46f-4c8a-8c9b-706a10fb0674 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:13.405604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212f16b-e46f-4c8a-8c9b-706a10fb0674 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:13.405888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212f16b-e46f-4c8a-8c9b-706a10fb0674 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:13.406760 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d212f16b-e46f-4c8a-8c9b-706a10fb0674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:13.407071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d212f16b-e46f-4c8a-8c9b-706a10fb0674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:13.408513 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d212f16b-e46f-4c8a-8c9b-706a10fb0674 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:13.421450 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5e7e0-1718-46db-b925-5f59c40bc644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:13.422632 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5e7e0-1718-46db-b925-5f59c40bc644 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:13.423142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5e7e0-1718-46db-b925-5f59c40bc644 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:13.423651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5e7e0-1718-46db-b925-5f59c40bc644 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:13.423888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 377/760] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:13.424756 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212f16b-e46f-4c8a-8c9b-706a10fb0674 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:13.438790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5e7e0-1718-46db-b925-5f59c40bc644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:13.440784 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5e7e0-1718-46db-b925-5f59c40bc644 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:13.452799 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:13.469681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-12b5e7e0-1718-46db-b925-5f59c40bc644 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:13.469681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:13.469681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:13.469681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:13.469681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:13.469681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:13.469681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:13.469681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:13.469681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:13.469681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:13.469681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:13.469681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:13.469681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:13.469681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:13.469681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:13.471160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:13.471160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:13.471160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:13.471160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:13.471160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:13.471160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:13.471160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:13.471160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:13.471160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:13.471160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:13.471160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:13.471160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:13.471160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:13.471160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:13.471160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:13.471160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:13.471160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:13.472672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:13.472672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:13.472672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:13.472672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:13.475480 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 384/761] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:13.477086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5e7e0-1718-46db-b925-5f59c40bc644 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:13.492893 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:13.559000 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212f16b-e46f-4c8a-8c9b-706a10fb0674 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:13.598080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5e7e0-1718-46db-b925-5f59c40bc644 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:13.599066 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1eef6-5cf0-47a9-99fb-dbb0e28b3ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:13.599902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1eef6-5cf0-47a9-99fb-dbb0e28b3ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:13.600190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1eef6-5cf0-47a9-99fb-dbb0e28b3ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:13.600470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1eef6-5cf0-47a9-99fb-dbb0e28b3ba3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:13.601252 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea1eef6-5cf0-47a9-99fb-dbb0e28b3ba3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:13.603134 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea1eef6-5cf0-47a9-99fb-dbb0e28b3ba3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:13.617193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-0ea1eef6-5cf0-47a9-99fb-dbb0e28b3ba3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 06:17:13.617193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:13.617193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:13.617193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:13.617193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:13.617193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:13.617193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:13.617193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:13.617193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 06:17:13.617193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:13.617193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:13.617193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:13.617193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:13.617193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:13.617193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:13.617193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:13.618279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:13.618279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:13.618279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:13.618279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:13.618279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:13.618279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:13.618279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:13.618279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:13.618279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:13.618279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:13.618279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:13.618279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:13.618279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:13.618279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:13.618279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:13.618279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:13.619143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:13.619143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:13.619143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:13.619143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:13.619143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:13.619143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:13.619143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:13.619143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:13.619143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:13.619143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:13.619143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:13.619143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:13.619143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:13.619143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 06:17:13.619143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:13.624192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 378/762] 10.222.0.65 () {56 vars in 1129 bytes} [Thu May 19 06:17:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:13.625308 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1eef6-5cf0-47a9-99fb-dbb0e28b3ba3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:13.646005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239af9e9-2a17-435e-917f-5d99b71e3605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:13.647232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239af9e9-2a17-435e-917f-5d99b71e3605 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:13.647735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239af9e9-2a17-435e-917f-5d99b71e3605 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:13.648207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239af9e9-2a17-435e-917f-5d99b71e3605 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:13.656869 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239af9e9-2a17-435e-917f-5d99b71e3605 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:13.658769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239af9e9-2a17-435e-917f-5d99b71e3605 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:13.674187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-239af9e9-2a17-435e-917f-5d99b71e3605 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:13.674187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:13.674187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:13.674187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:13.674187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:13.674187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:13.674187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:13.674187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:13.674187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:13.674187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:13.674187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:13.674187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:13.674187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:13.674187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:13.674187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:13.675117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:13.675117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:13.675117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:13.675117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:13.675117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:13.675117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:13.675117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:13.675117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:13.675117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:13.675117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:13.675117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:13.675117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:13.675117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:13.675117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:13.675117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:13.675117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:13.675117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:13.676068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:13.676068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:13.676068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:13.676068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:13.676068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:13.676068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:13.676068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:13.676068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:13.678685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 385/763] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:13.679886 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239af9e9-2a17-435e-917f-5d99b71e3605 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:13.739310 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1eef6-5cf0-47a9-99fb-dbb0e28b3ba3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:13.750396 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:13.754035 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:13.784896 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f80cf-a039-4004-b33e-39533189e336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:13.785881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f80cf-a039-4004-b33e-39533189e336 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:13.786426 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f80cf-a039-4004-b33e-39533189e336 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:13.786864 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f80cf-a039-4004-b33e-39533189e336 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:13.787910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f80cf-a039-4004-b33e-39533189e336 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:13.788400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f80cf-a039-4004-b33e-39533189e336 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:13.790015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f80cf-a039-4004-b33e-39533189e336 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:13.809645 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 379/764] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:13.809841 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239af9e9-2a17-435e-917f-5d99b71e3605 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:13.810968 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f80cf-a039-4004-b33e-39533189e336 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:13.852865 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa5d53-7d76-4457-9508-39e2c951610f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:13.853671 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa5d53-7d76-4457-9508-39e2c951610f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:13.853963 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa5d53-7d76-4457-9508-39e2c951610f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:13.854366 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa5d53-7d76-4457-9508-39e2c951610f None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:13.855104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa5d53-7d76-4457-9508-39e2c951610f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:13.855430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa5d53-7d76-4457-9508-39e2c951610f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:13.856923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa5d53-7d76-4457-9508-39e2c951610f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:13.867316 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 386/765] 10.222.0.65 () {56 vars in 1124 bytes} [Thu May 19 06:17:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:13.867316 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa5d53-7d76-4457-9508-39e2c951610f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:13.933877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f80cf-a039-4004-b33e-39533189e336 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:13.972878 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8611cef-39e4-4a69-b240-33788ea5e450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:13.973829 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8611cef-39e4-4a69-b240-33788ea5e450 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:13.974187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8611cef-39e4-4a69-b240-33788ea5e450 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:13.974538 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8611cef-39e4-4a69-b240-33788ea5e450 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:13.975502 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8611cef-39e4-4a69-b240-33788ea5e450 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:13.975880 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8611cef-39e4-4a69-b240-33788ea5e450 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:13.977452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8611cef-39e4-4a69-b240-33788ea5e450 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:13.991096 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 380/766] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:17:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:13.991953 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8611cef-39e4-4a69-b240-33788ea5e450 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:13.997472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa5d53-7d76-4457-9508-39e2c951610f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:14.026636 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:14.028245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:14.043670 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3bb74-7734-4850-b8e7-bdc620ddd5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:14.045097 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3bb74-7734-4850-b8e7-bdc620ddd5fe None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:14.045525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3bb74-7734-4850-b8e7-bdc620ddd5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:14.045923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3bb74-7734-4850-b8e7-bdc620ddd5fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:14.054570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3bb74-7734-4850-b8e7-bdc620ddd5fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:14.056562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3bb74-7734-4850-b8e7-bdc620ddd5fe None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:14.076025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-46e3bb74-7734-4850-b8e7-bdc620ddd5fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:14.076025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:14.076025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:14.076025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:14.076025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:14.076025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:14.076025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:14.076025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:14.076025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:14.076025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:14.076025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:14.076025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:14.076025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:14.076025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:14.076025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:14.077211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:14.077211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:14.077211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:14.077211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:14.077211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:14.077211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:14.077211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:14.077211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:14.077211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:14.077211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:14.077211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:14.077211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:14.077211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:14.077211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:14.077211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:14.077211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:14.077211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:14.078474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:14.078474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:14.078474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:14.078474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:14.078474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:14.078474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:14.078474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:14.078474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:14.079999 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 387/767] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:14.080799 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3bb74-7734-4850-b8e7-bdc620ddd5fe None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:14.122663 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8611cef-39e4-4a69-b240-33788ea5e450 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:14.163843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26814047-0acd-4030-b83d-82d7ec3e67cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:14.164520 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26814047-0acd-4030-b83d-82d7ec3e67cb None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:14.164810 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26814047-0acd-4030-b83d-82d7ec3e67cb None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:14.165078 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26814047-0acd-4030-b83d-82d7ec3e67cb None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:14.165825 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26814047-0acd-4030-b83d-82d7ec3e67cb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:14.166130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26814047-0acd-4030-b83d-82d7ec3e67cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:14.168062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26814047-0acd-4030-b83d-82d7ec3e67cb None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:14.173217 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 381/768] 10.222.0.65 () {56 vars in 1124 bytes} [Thu May 19 06:17:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:14.174058 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26814047-0acd-4030-b83d-82d7ec3e67cb None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:14.228436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3bb74-7734-4850-b8e7-bdc620ddd5fe None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:14.268033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7dcdd-fb84-42e6-8e6e-83a20b37222f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:14.268956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7dcdd-fb84-42e6-8e6e-83a20b37222f None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:14.269299 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7dcdd-fb84-42e6-8e6e-83a20b37222f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:14.269571 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7dcdd-fb84-42e6-8e6e-83a20b37222f None admin] PATH_INFO: `/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/2a09f4878eac4d60a9a7dcedba15c05d/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:14.272094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7dcdd-fb84-42e6-8e6e-83a20b37222f None admin] RBAC: Authorizing `identity:create_grant(project_id=a3d399dc95424b848b736c0d0298d9a9, user_id=2a09f4878eac4d60a9a7dcedba15c05d, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:14.273144 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7dcdd-fb84-42e6-8e6e-83a20b37222f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:14.290169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 388/769] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:17:14 2022] PUT /identity/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/2a09f4878eac4d60a9a7dcedba15c05d/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:17:14.291996 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3f077-4c05-404d-8cf6-0ae790729b0e None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:14.292543 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3f077-4c05-404d-8cf6-0ae790729b0e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:14.293060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3f077-4c05-404d-8cf6-0ae790729b0e None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:14.294711 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 389/770] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:14.295753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3f077-4c05-404d-8cf6-0ae790729b0e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:14.330089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:14.354177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26814047-0acd-4030-b83d-82d7ec3e67cb None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:14.386076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:14.438150 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf6574-0dbf-47ef-9c23-50bdd30558fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:14.439321 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf6574-0dbf-47ef-9c23-50bdd30558fd None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:14.439800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf6574-0dbf-47ef-9c23-50bdd30558fd None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:14.440244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf6574-0dbf-47ef-9c23-50bdd30558fd None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:14.441846 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf6574-0dbf-47ef-9c23-50bdd30558fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:14.442402 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf6574-0dbf-47ef-9c23-50bdd30558fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:14.445235 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf6574-0dbf-47ef-9c23-50bdd30558fd None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:14.471001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 382/771] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:17:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:14.473440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ac51c-2874-4b84-952f-52b44c75385b None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:14.473959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ac51c-2874-4b84-952f-52b44c75385b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:14.474610 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ac51c-2874-4b84-952f-52b44c75385b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:14.483992 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3f077-4c05-404d-8cf6-0ae790729b0e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:14.504511 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-882ac51c-2874-4b84-952f-52b44c75385b None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:14.506787 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-882ac51c-2874-4b84-952f-52b44c75385b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:14.550011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83883240-a601-4da4-9148-2007a2fa156e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:14.550668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83883240-a601-4da4-9148-2007a2fa156e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:14.550972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83883240-a601-4da4-9148-2007a2fa156e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:14.550972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83883240-a601-4da4-9148-2007a2fa156e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:14.552055 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83883240-a601-4da4-9148-2007a2fa156e None admin] RBAC: Adding query filter params (interface=public, service_id=ca8ce3d39763435ea6bbc93348a5491a, region_id=RegionOne) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:14.552391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83883240-a601-4da4-9148-2007a2fa156e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:14.553832 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83883240-a601-4da4-9148-2007a2fa156e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:14.563877 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 390/772] 10.222.0.65 () {56 vars in 1251 bytes} [Thu May 19 06:17:14 2022] GET /identity/v3/endpoints?service_id=ca8ce3d39763435ea6bbc93348a5491a&interface=public®ion_id=RegionOne => generated 194 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:14.565858 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83883240-a601-4da4-9148-2007a2fa156e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:14.634874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 383/773] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:14.637650 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ac51c-2874-4b84-952f-52b44c75385b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:14.685037 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83883240-a601-4da4-9148-2007a2fa156e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:14.700692 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:14.734713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b6c4a-324c-4215-8045-18c48b8dc0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:14.735517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b6c4a-324c-4215-8045-18c48b8dc0ac None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:14.735992 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b6c4a-324c-4215-8045-18c48b8dc0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:14.736464 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b6c4a-324c-4215-8045-18c48b8dc0ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:14.737516 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b6c4a-324c-4215-8045-18c48b8dc0ac None admin] RBAC: Adding query filter params (role.id=5cc9ca1bc48b4d749d21d7111b14e187, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=aee77a3ed91445ab8b7f58e7f8bc348a) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:14.737975 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b6c4a-324c-4215-8045-18c48b8dc0ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:14.739713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b6c4a-324c-4215-8045-18c48b8dc0ac None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:14.749322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 391/774] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:17:14 2022] GET /identity/v3/role_assignments?user.id=aee77a3ed91445ab8b7f58e7f8bc348a&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=5cc9ca1bc48b4d749d21d7111b14e187 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:14.753822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b1a44-9dba-41e5-9411-2934c49cbda7 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:14.754218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b1a44-9dba-41e5-9411-2934c49cbda7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:14.754396 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b1a44-9dba-41e5-9411-2934c49cbda7 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:14.756107 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 392/775] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:14.756825 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b1a44-9dba-41e5-9411-2934c49cbda7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:14.764943 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:14.802196 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ac51c-2874-4b84-952f-52b44c75385b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:14.873275 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbf955-0823-40c2-96a8-3d4abbb4da4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:14.874565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbf955-0823-40c2-96a8-3d4abbb4da4b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:14.875073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbf955-0823-40c2-96a8-3d4abbb4da4b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:14.875561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbf955-0823-40c2-96a8-3d4abbb4da4b None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:14.876794 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbf955-0823-40c2-96a8-3d4abbb4da4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:14.879263 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbf955-0823-40c2-96a8-3d4abbb4da4b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:14.897969 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 384/776] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 06:17:14 2022] GET /identity/v3/services => generated 772 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:14.900754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ddcbc-b369-436c-ae2b-2aaa7cadcca3 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:14.901586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ddcbc-b369-436c-ae2b-2aaa7cadcca3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:14.905188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ddcbc-b369-436c-ae2b-2aaa7cadcca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:14.938573 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b1a44-9dba-41e5-9411-2934c49cbda7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:14.971296 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5f2ddcbc-b369-436c-ae2b-2aaa7cadcca3 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:14.971942 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5f2ddcbc-b369-436c-ae2b-2aaa7cadcca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:15.011583 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d0dad-d21d-4ad6-bcbb-ce7ffa2f2912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:15.012535 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d0dad-d21d-4ad6-bcbb-ce7ffa2f2912 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:15.012823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d0dad-d21d-4ad6-bcbb-ce7ffa2f2912 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:15.013133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d0dad-d21d-4ad6-bcbb-ce7ffa2f2912 None admin] PATH_INFO: `/v3/services/network` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:15.014004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d0dad-d21d-4ad6-bcbb-ce7ffa2f2912 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:15.015477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d0dad-d21d-4ad6-bcbb-ce7ffa2f2912 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:15.032562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-ed9d0dad-d21d-4ad6-bcbb-ce7ffa2f2912 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 06:17:15.032562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:15.032562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:15.032562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:15.032562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:15.032562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:15.032562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:15.032562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:15.032562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 06:17:15.032562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:15.032562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:15.032562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:15.032562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:15.032562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:15.032562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:15.032562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:15.032562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:15.034062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:15.034062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:15.034062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:15.034062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:15.034062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:15.034062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:15.034062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:15.034062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:15.034062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:15.034062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:15.034062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:15.034062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:15.034062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:15.034062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:15.034062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:15.034062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:15.034890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:15.034890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:15.034890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:15.034890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:15.034890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:15.034890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:15.034890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:15.034890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:15.034890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:15.034890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:15.034890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:15.034890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:15.034890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 06:17:15.034890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:15.034890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 393/777] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:14 2022] GET /identity/v3/services/network => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:15.044315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d0dad-d21d-4ad6-bcbb-ce7ffa2f2912 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:15.128076 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 385/778] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:15.134959 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:15.167759 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ddcbc-b369-436c-ae2b-2aaa7cadcca3 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:15.248519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d0dad-d21d-4ad6-bcbb-ce7ffa2f2912 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:15.300219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f1d8f-e028-4e33-86bc-6238d19e59c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:15.300657 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f1d8f-e028-4e33-86bc-6238d19e59c8 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:15.300946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f1d8f-e028-4e33-86bc-6238d19e59c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:15.301230 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f1d8f-e028-4e33-86bc-6238d19e59c8 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:15.302012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f1d8f-e028-4e33-86bc-6238d19e59c8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:15.302310 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f1d8f-e028-4e33-86bc-6238d19e59c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:15.303771 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f1d8f-e028-4e33-86bc-6238d19e59c8 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:15.305388 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:15.329641 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 394/779] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 06:17:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:15.341315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f1d8f-e028-4e33-86bc-6238d19e59c8 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:15.343595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ddcbc-b369-436c-ae2b-2aaa7cadcca3 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:15.411015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03cb2fb-52d0-4463-b0be-f1a021ff2334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:15.412257 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cb2fb-52d0-4463-b0be-f1a021ff2334 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:15.412767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cb2fb-52d0-4463-b0be-f1a021ff2334 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:15.413275 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cb2fb-52d0-4463-b0be-f1a021ff2334 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:15.442137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03cb2fb-52d0-4463-b0be-f1a021ff2334 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:17:15.455827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-c03cb2fb-52d0-4463-b0be-f1a021ff2334 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:17:15.455827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:15.455827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:15.455827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:15.455827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:15.455827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:15.455827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:15.455827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:15.455827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:15.455827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:15.455827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:15.455827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:15.455827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:17:15.455827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:17:15.455827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:17:15.455827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:17:15.456838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:17:15.456838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:17:15.456838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:17:15.456838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:17:15.456838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:15.456838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:15.456838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:15.456838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:15.456838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:15.456838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:15.456838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:15.456838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:15.456838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:15.456838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:15.456838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:15.456838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:15.458129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:15.458129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:15.458129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:15.458129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:15.458129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:17:15.458129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:17:15.458129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:17:15.458129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:17:15.458129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:17:15.458129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:17:15.458129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:17:15.458129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:15.464152 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 386/780] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:17:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:15.474742 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03cb2fb-52d0-4463-b0be-f1a021ff2334 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:15.601842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03cb2fb-52d0-4463-b0be-f1a021ff2334 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:15.641003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651bdf9-4ec2-4ba2-af0d-43fc999e2557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:15.642069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651bdf9-4ec2-4ba2-af0d-43fc999e2557 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:15.642556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651bdf9-4ec2-4ba2-af0d-43fc999e2557 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:15.643028 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651bdf9-4ec2-4ba2-af0d-43fc999e2557 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:15.644086 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651bdf9-4ec2-4ba2-af0d-43fc999e2557 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:15.644596 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651bdf9-4ec2-4ba2-af0d-43fc999e2557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:15.646258 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651bdf9-4ec2-4ba2-af0d-43fc999e2557 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:15.653680 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:15.657945 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 387/781] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:17:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:15.663304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:15.665261 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651bdf9-4ec2-4ba2-af0d-43fc999e2557 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:15.675313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f1d8f-e028-4e33-86bc-6238d19e59c8 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:15.752914 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1c87b-cd39-4297-b9bd-78f5c328b09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:15.754152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1c87b-cd39-4297-b9bd-78f5c328b09a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:15.754641 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1c87b-cd39-4297-b9bd-78f5c328b09a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:15.755143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1c87b-cd39-4297-b9bd-78f5c328b09a None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:15.756070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1c87b-cd39-4297-b9bd-78f5c328b09a None admin] RBAC: Adding query filter params (type=network) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:15.756576 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1c87b-cd39-4297-b9bd-78f5c328b09a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:15.758191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1c87b-cd39-4297-b9bd-78f5c328b09a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:15.776739 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 395/782] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 06:17:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:15.789837 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1c87b-cd39-4297-b9bd-78f5c328b09a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:15.897145 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651bdf9-4ec2-4ba2-af0d-43fc999e2557 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:16.003640 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1c87b-cd39-4297-b9bd-78f5c328b09a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:16.004742 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e84b1-a865-41f2-9426-a358fb1dbd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:16.005828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e84b1-a865-41f2-9426-a358fb1dbd35 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:16.006474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e84b1-a865-41f2-9426-a358fb1dbd35 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:16.007361 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e84b1-a865-41f2-9426-a358fb1dbd35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:16.046665 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e84b1-a865-41f2-9426-a358fb1dbd35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:16.049269 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e84b1-a865-41f2-9426-a358fb1dbd35 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:16.072393 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7283317-7637-4abd-b857-7ab52c8b4cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:16.074137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7283317-7637-4abd-b857-7ab52c8b4cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:16.074629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7283317-7637-4abd-b857-7ab52c8b4cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:16.075104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7283317-7637-4abd-b857-7ab52c8b4cb3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:16.076122 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7283317-7637-4abd-b857-7ab52c8b4cb3 None admin] RBAC: Adding query filter params (interface=public, service_id=23a0dbb383824fadacc20902dae2df8c, region_id=RegionOne) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:16.076655 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7283317-7637-4abd-b857-7ab52c8b4cb3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:16.081601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7283317-7637-4abd-b857-7ab52c8b4cb3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:16.087400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-347e84b1-a865-41f2-9426-a358fb1dbd35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:16.087400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:16.087400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:16.087400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:16.087400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:16.087400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:16.087400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:16.087400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:16.087400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:16.087400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:16.087400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:16.087400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:16.087400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:16.087400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:16.087400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:16.088958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:16.088958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:16.088958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:16.088958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:16.088958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:16.088958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:16.088958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:16.088958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:16.088958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:16.088958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:16.088958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:16.088958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:16.088958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:16.088958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:16.088958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:16.088958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:16.088958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:16.089890 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:16.089890 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:16.089890 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:16.089890 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:16.096232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 396/784] 10.222.0.65 () {56 vars in 1251 bytes} [Thu May 19 06:17:15 2022] GET /identity/v3/endpoints?service_id=23a0dbb383824fadacc20902dae2df8c&interface=public®ion_id=RegionOne => generated 194 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:16.096582 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 388/784] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:16.101877 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7283317-7637-4abd-b857-7ab52c8b4cb3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:16.105675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e84b1-a865-41f2-9426-a358fb1dbd35 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:16.168371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:16.305681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:16.367084 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7283317-7637-4abd-b857-7ab52c8b4cb3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:16.444350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d60dc7-8dd9-40fc-9d40-9276faf605fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:16.445449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d60dc7-8dd9-40fc-9d40-9276faf605fc None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:16.445956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d60dc7-8dd9-40fc-9d40-9276faf605fc None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:16.446456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d60dc7-8dd9-40fc-9d40-9276faf605fc None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:16.448367 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d60dc7-8dd9-40fc-9d40-9276faf605fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:16.448926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d60dc7-8dd9-40fc-9d40-9276faf605fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:16.451572 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d60dc7-8dd9-40fc-9d40-9276faf605fc None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:16.462338 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e84b1-a865-41f2-9426-a358fb1dbd35 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:16.511826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 397/785] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:16.521772 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d60dc7-8dd9-40fc-9d40-9276faf605fc None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:16.573870 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38cfe5-61fa-4b74-ac5a-71a734632611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:16.575027 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38cfe5-61fa-4b74-ac5a-71a734632611 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:16.575523 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38cfe5-61fa-4b74-ac5a-71a734632611 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:16.576019 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38cfe5-61fa-4b74-ac5a-71a734632611 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:16.579472 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38cfe5-61fa-4b74-ac5a-71a734632611 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:16.582070 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38cfe5-61fa-4b74-ac5a-71a734632611 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:16.606763 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 389/786] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 06:17:16 2022] GET /identity/v3/services => generated 772 bytes in 502 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:16.791097 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:16.826745 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d60dc7-8dd9-40fc-9d40-9276faf605fc None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:16.903908 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d189a-9786-455f-a7dc-add76e1dccba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:16.905632 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d189a-9786-455f-a7dc-add76e1dccba None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:16.906170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d189a-9786-455f-a7dc-add76e1dccba None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:16.906686 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d189a-9786-455f-a7dc-add76e1dccba None admin] PATH_INFO: `/v3/users/swift` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:16.921164 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d189a-9786-455f-a7dc-add76e1dccba None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:16.923950 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d189a-9786-455f-a7dc-add76e1dccba None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:16.954767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-748d189a-9786-455f-a7dc-add76e1dccba None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 06:17:16.954767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:16.954767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:16.954767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:16.954767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:16.954767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:16.954767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:16.954767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:16.954767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:16.954767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:16.954767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:16.954767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:16.954767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:17:16.954767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:17:16.954767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:17:16.954767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:16.955976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:16.957155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:16.957155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:16.957155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:17:16.957155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:17:16.957155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:17:16.957155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:17:16.957155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:17:16.957155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:17:16.957155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 06:17:16.957155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:16.961717 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 398/787] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:17:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 437 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:16.967811 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38cfe5-61fa-4b74-ac5a-71a734632611 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:17.282818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38cfe5-61fa-4b74-ac5a-71a734632611 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:17.324003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:17.394632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf346ca7-52fa-448f-a7e7-9184ed24d297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:17.395337 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf346ca7-52fa-448f-a7e7-9184ed24d297 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:17.395632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf346ca7-52fa-448f-a7e7-9184ed24d297 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:17.395905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf346ca7-52fa-448f-a7e7-9184ed24d297 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:17.396760 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf346ca7-52fa-448f-a7e7-9184ed24d297 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:17.397071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf346ca7-52fa-448f-a7e7-9184ed24d297 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:17.398551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf346ca7-52fa-448f-a7e7-9184ed24d297 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:17.450067 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 390/788] 10.222.0.65 () {56 vars in 1135 bytes} [Thu May 19 06:17:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 483 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:17.461870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d189a-9786-455f-a7dc-add76e1dccba None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:17.723823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d189a-9786-455f-a7dc-add76e1dccba None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:17.806121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcc29b-81b0-45f2-a7fe-8319c4eb002c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:17.807991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcc29b-81b0-45f2-a7fe-8319c4eb002c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:17.808395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcc29b-81b0-45f2-a7fe-8319c4eb002c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:17.808679 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcc29b-81b0-45f2-a7fe-8319c4eb002c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:17.830498 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcc29b-81b0-45f2-a7fe-8319c4eb002c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:17.831988 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcc29b-81b0-45f2-a7fe-8319c4eb002c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:17.852156 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:17.856704 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e0bf3-7183-4418-8598-20116e468317 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:17.857449 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e0bf3-7183-4418-8598-20116e468317 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:17.857748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e0bf3-7183-4418-8598-20116e468317 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:17.860827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 391/789] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:17.863305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-fcdcc29b-81b0-45f2-a7fe-8319c4eb002c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:17.863305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:17.863305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:17.863305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:17.863305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:17.863305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:17.863305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:17.863305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:17.863305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:17.863305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:17.863305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:17.863305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:17.863305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:17.863305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:17.863305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:17.866327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:17.866327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:17.866327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:17.866327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:17.866327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:17.866327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:17.866327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:17.866327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:17.866327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:17.866327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:17.866327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:17.866327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:17.866327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:17.866327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:17.866327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:17.866327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:17.866327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:17.868155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:17.868155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:17.868155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:17.868155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:17.868155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 399/790] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:17.868710 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1e2a2-aa02-47fd-bb72-038409e3f976 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:17.868710 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1e2a2-aa02-47fd-bb72-038409e3f976 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:17.868710 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1e2a2-aa02-47fd-bb72-038409e3f976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:17.872226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcc29b-81b0-45f2-a7fe-8319c4eb002c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:17.939210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-8ad1e2a2-aa02-47fd-bb72-038409e3f976 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:17.941509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8ad1e2a2-aa02-47fd-bb72-038409e3f976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:17.998908 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcc29b-81b0-45f2-a7fe-8319c4eb002c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:18.038531 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16d8c8-3930-4ba1-8641-d48a2e90cc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:18.039503 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16d8c8-3930-4ba1-8641-d48a2e90cc82 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:18.039946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16d8c8-3930-4ba1-8641-d48a2e90cc82 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:18.040378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16d8c8-3930-4ba1-8641-d48a2e90cc82 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:18.041385 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16d8c8-3930-4ba1-8641-d48a2e90cc82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:18.042072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16d8c8-3930-4ba1-8641-d48a2e90cc82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:18.044006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16d8c8-3930-4ba1-8641-d48a2e90cc82 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:18.059945 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 400/791] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:18.065886 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16d8c8-3930-4ba1-8641-d48a2e90cc82 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:18.134360 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:18.153777 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 392/792] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:18.155831 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511366f2-8b61-4944-a0d9-290a6d226366 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:18.156169 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511366f2-8b61-4944-a0d9-290a6d226366 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:18.156450 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511366f2-8b61-4944-a0d9-290a6d226366 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:18.165305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 393/793] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:18.165305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b261fc-e683-44f7-83e2-41a3e5c0fec8 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:18.165305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b261fc-e683-44f7-83e2-41a3e5c0fec8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:18.165305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b261fc-e683-44f7-83e2-41a3e5c0fec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:18.192664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16d8c8-3930-4ba1-8641-d48a2e90cc82 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:18.193901 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-24b261fc-e683-44f7-83e2-41a3e5c0fec8 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:18.195275 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-24b261fc-e683-44f7-83e2-41a3e5c0fec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:18.232740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695d6ab-ecb5-4d46-a276-9a38faacb921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:18.233894 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695d6ab-ecb5-4d46-a276-9a38faacb921 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:18.234380 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695d6ab-ecb5-4d46-a276-9a38faacb921 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:18.234774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695d6ab-ecb5-4d46-a276-9a38faacb921 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:18.242655 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695d6ab-ecb5-4d46-a276-9a38faacb921 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:18.245094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695d6ab-ecb5-4d46-a276-9a38faacb921 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:18.268557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-a695d6ab-ecb5-4d46-a276-9a38faacb921 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:18.268557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:18.268557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:18.268557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:18.268557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:18.268557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:18.268557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:18.268557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:18.268557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:18.268557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:18.268557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:18.268557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:18.268557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:18.268557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:18.268557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:18.269629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:18.269629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:18.269629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:18.269629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:18.269629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:18.269629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:18.269629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:18.269629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:18.269629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:18.269629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:18.269629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:18.269629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:18.269629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:18.269629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:18.269629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:18.269629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:18.269629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:18.270736 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:18.270736 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:18.270736 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:18.270736 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:18.270736 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:18.270736 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:18.270736 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:18.270736 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:18.270736 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 401/794] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:18.271579 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695d6ab-ecb5-4d46-a276-9a38faacb921 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:18.313421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 394/795] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:18.313421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b261fc-e683-44f7-83e2-41a3e5c0fec8 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:18.389981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:18.449545 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695d6ab-ecb5-4d46-a276-9a38faacb921 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:18.480430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:18.486434 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b261fc-e683-44f7-83e2-41a3e5c0fec8 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:18.501646 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5e3c6-2f6e-40bd-bad0-17083f245a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:18.502668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5e3c6-2f6e-40bd-bad0-17083f245a50 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:18.503126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5e3c6-2f6e-40bd-bad0-17083f245a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:18.503673 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5e3c6-2f6e-40bd-bad0-17083f245a50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:18.521754 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5e3c6-2f6e-40bd-bad0-17083f245a50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:17:18.540542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-71b5e3c6-2f6e-40bd-bad0-17083f245a50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:17:18.540542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:18.540542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:18.540542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:18.540542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:18.540542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:18.540542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:18.540542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:18.540542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:18.540542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:18.540542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:18.540542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:18.540542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:17:18.540542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:17:18.540542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:17:18.540542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:17:18.542621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:17:18.542621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:17:18.542621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:17:18.542621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:17:18.542621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:18.542621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:18.542621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:18.542621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:18.542621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:18.542621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:18.542621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:18.542621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:18.542621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:18.542621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:18.542621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:18.542621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:18.544136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:18.544136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:18.544136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:18.544136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:18.544136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:17:18.544136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:17:18.544136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:17:18.544136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:17:18.544136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:17:18.544136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:17:18.544136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:17:18.544136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:18.547356 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f26b60-30b6-49fa-b08e-6847ddf288a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:18.547752 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 402/796] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:17:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:18.548447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f26b60-30b6-49fa-b08e-6847ddf288a2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:18.548716 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5e3c6-2f6e-40bd-bad0-17083f245a50 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:18.549043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f26b60-30b6-49fa-b08e-6847ddf288a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:18.549611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f26b60-30b6-49fa-b08e-6847ddf288a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:18.550586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f26b60-30b6-49fa-b08e-6847ddf288a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:18.550898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f26b60-30b6-49fa-b08e-6847ddf288a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:18.552391 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f26b60-30b6-49fa-b08e-6847ddf288a2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:18.566171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 395/797] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:17:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:18.567550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ed891-2a0a-4c23-a19c-60c254d4596e None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:18.567842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ed891-2a0a-4c23-a19c-60c254d4596e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:18.568123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ed891-2a0a-4c23-a19c-60c254d4596e None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:18.569679 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 396/798] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:18.570281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ed891-2a0a-4c23-a19c-60c254d4596e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:18.689493 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5e3c6-2f6e-40bd-bad0-17083f245a50 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:18.699731 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:18.710993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ed891-2a0a-4c23-a19c-60c254d4596e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:18.732529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551069b9-19c8-4265-92d2-7c03bf30b4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:18.733381 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551069b9-19c8-4265-92d2-7c03bf30b4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:18.733674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551069b9-19c8-4265-92d2-7c03bf30b4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:18.733955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551069b9-19c8-4265-92d2-7c03bf30b4a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:18.748236 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551069b9-19c8-4265-92d2-7c03bf30b4a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:17:18.749271 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c758cf-3819-472d-b738-f0d123ded79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:18.750008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c758cf-3819-472d-b738-f0d123ded79a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:18.750295 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c758cf-3819-472d-b738-f0d123ded79a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:18.750586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c758cf-3819-472d-b738-f0d123ded79a None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:18.751339 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c758cf-3819-472d-b738-f0d123ded79a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:18.751644 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c758cf-3819-472d-b738-f0d123ded79a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:18.753073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c758cf-3819-472d-b738-f0d123ded79a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:18.756513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-551069b9-19c8-4265-92d2-7c03bf30b4a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:17:18.756513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:18.756513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:18.756513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:18.756513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:18.756513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:18.756513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:18.756513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:18.756513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:18.756513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:18.756513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:18.756513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:18.756513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:17:18.756513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:17:18.756513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:17:18.756513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:17:18.757805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:17:18.757805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:17:18.757805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:17:18.757805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:17:18.757805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:18.757805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:18.757805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:18.757805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:18.757805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:18.757805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:18.757805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:18.757805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:18.757805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:18.757805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:18.757805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:18.757805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:18.758775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:18.758775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:18.758775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:18.758775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:18.758775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:17:18.758775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:17:18.758775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:17:18.758775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:17:18.758775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:17:18.758775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:17:18.758775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:17:18.758775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:18.763738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 403/799] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:17:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:18.769205 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 397/800] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:17:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:18.769581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49252b8c-d021-48d2-89d3-e3bbbc1987ac None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:18.769768 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c758cf-3819-472d-b738-f0d123ded79a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:18.769939 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49252b8c-d021-48d2-89d3-e3bbbc1987ac None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:18.769939 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49252b8c-d021-48d2-89d3-e3bbbc1987ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:18.786356 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-49252b8c-d021-48d2-89d3-e3bbbc1987ac None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:18.787498 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-49252b8c-d021-48d2-89d3-e3bbbc1987ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:18.828250 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:18.872804 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 404/801] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:18.873751 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49252b8c-d021-48d2-89d3-e3bbbc1987ac None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:18.892089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c758cf-3819-472d-b738-f0d123ded79a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:18.934430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42c0d7-ec64-400c-817c-69a38cdd6897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:18.935328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42c0d7-ec64-400c-817c-69a38cdd6897 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:18.935610 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42c0d7-ec64-400c-817c-69a38cdd6897 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:18.935887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42c0d7-ec64-400c-817c-69a38cdd6897 None admin] PATH_INFO: `/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/5fc205d053d54506bb6c9227e4a1b044/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:18.938093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42c0d7-ec64-400c-817c-69a38cdd6897 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3d399dc95424b848b736c0d0298d9a9, user_id=5fc205d053d54506bb6c9227e4a1b044, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:18.939622 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42c0d7-ec64-400c-817c-69a38cdd6897 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:18.961319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 398/802] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:17:18 2022] PUT /identity/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/5fc205d053d54506bb6c9227e4a1b044/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:17:18.961319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42c0d7-ec64-400c-817c-69a38cdd6897 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:18.980418 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:19.016889 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49252b8c-d021-48d2-89d3-e3bbbc1987ac None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:19.063471 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e74073-23ad-4bee-86d4-0ccbbdc534a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:19.064133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e74073-23ad-4bee-86d4-0ccbbdc534a7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:19.064416 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e74073-23ad-4bee-86d4-0ccbbdc534a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:19.064690 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e74073-23ad-4bee-86d4-0ccbbdc534a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:19.065518 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e74073-23ad-4bee-86d4-0ccbbdc534a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:19.065841 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e74073-23ad-4bee-86d4-0ccbbdc534a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:19.067588 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e74073-23ad-4bee-86d4-0ccbbdc534a7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:19.081924 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 405/803] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:17:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:19.082891 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e74073-23ad-4bee-86d4-0ccbbdc534a7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:19.110074 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42c0d7-ec64-400c-817c-69a38cdd6897 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:19.129494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:19.160798 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346652ea-922c-48a1-a431-0289446abfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:19.162039 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346652ea-922c-48a1-a431-0289446abfce None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:19.162562 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346652ea-922c-48a1-a431-0289446abfce None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:19.163166 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346652ea-922c-48a1-a431-0289446abfce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:19.175355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346652ea-922c-48a1-a431-0289446abfce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:19.177242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346652ea-922c-48a1-a431-0289446abfce None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:19.198775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-346652ea-922c-48a1-a431-0289446abfce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:19.198775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:19.198775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:19.198775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:19.198775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:19.198775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:19.198775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:19.198775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:19.198775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:19.198775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:19.198775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:19.198775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:19.198775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:19.198775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:19.198775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:19.199738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:19.199738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:19.199738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:19.199738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:19.199738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:19.199738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:19.199738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:19.199738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:19.199738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:19.199738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:19.199738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:19.199738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:19.199738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:19.199738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:19.199738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:19.199738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:19.199738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:19.200627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:19.200627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:19.200627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:19.200627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:19.200627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 399/804] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:19.200897 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346652ea-922c-48a1-a431-0289446abfce None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:19.224895 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e74073-23ad-4bee-86d4-0ccbbdc534a7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:19.267892 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c708c-4c1b-4e22-bf4d-69ed8cdb6faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:19.268862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c708c-4c1b-4e22-bf4d-69ed8cdb6faf None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:19.269156 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c708c-4c1b-4e22-bf4d-69ed8cdb6faf None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:19.269476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c708c-4c1b-4e22-bf4d-69ed8cdb6faf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:19.270443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c708c-4c1b-4e22-bf4d-69ed8cdb6faf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:19.271929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c708c-4c1b-4e22-bf4d-69ed8cdb6faf None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:19.285599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:19.286544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-5f0c708c-4c1b-4e22-bf4d-69ed8cdb6faf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 06:17:19.286544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:19.286544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:19.286544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:19.286544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:19.286544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:19.286544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:19.286544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:19.286544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 06:17:19.286544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:19.286544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:19.286544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:19.286544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:19.286544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:19.286544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:19.286544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:19.287473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:19.287473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:19.287473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:19.287473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:19.287473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:19.287473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:19.287473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:19.287473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:19.287473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:19.287473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:19.287473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:19.287473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:19.287473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:19.287473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:19.287473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:19.287473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:19.288304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:19.288304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:19.288304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:19.288304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:19.288304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:19.288304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:19.288304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:19.288304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:19.288304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:19.288304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:19.288304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:19.288304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:19.288304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:19.288304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 06:17:19.288304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:19.288304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 406/805] 10.222.0.65 () {56 vars in 1129 bytes} [Thu May 19 06:17:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:19.289170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c708c-4c1b-4e22-bf4d-69ed8cdb6faf None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:19.338912 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346652ea-922c-48a1-a431-0289446abfce None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:19.380363 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567a580-bbfe-42ab-b8d0-6682863cc175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:19.381165 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567a580-bbfe-42ab-b8d0-6682863cc175 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:19.381493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567a580-bbfe-42ab-b8d0-6682863cc175 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:19.381926 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567a580-bbfe-42ab-b8d0-6682863cc175 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:19.394010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567a580-bbfe-42ab-b8d0-6682863cc175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:19.395611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567a580-bbfe-42ab-b8d0-6682863cc175 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:19.416191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-2567a580-bbfe-42ab-b8d0-6682863cc175 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:19.416191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:19.416191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:19.416191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:19.416191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:19.416191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:19.416191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:19.416191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:19.416191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:19.416191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:19.416191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:19.416191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:19.416191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:19.416191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:19.416191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:19.418410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:19.418410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:19.418410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:19.418410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:19.418410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:19.418410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:19.418410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:19.418410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:19.418410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:19.418410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:19.418410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:19.418410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:19.418410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:19.418410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:19.418410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:19.418410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:19.418410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:19.419371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:19.419371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:19.419371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:19.419371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:19.423066 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 400/806] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:19.424035 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567a580-bbfe-42ab-b8d0-6682863cc175 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:19.425830 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:19.430615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c708c-4c1b-4e22-bf4d-69ed8cdb6faf None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:19.481621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0805d6-2832-44ba-ab2a-23672025a1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:19.482809 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0805d6-2832-44ba-ab2a-23672025a1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:19.483095 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0805d6-2832-44ba-ab2a-23672025a1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:19.483380 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0805d6-2832-44ba-ab2a-23672025a1e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:19.484565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0805d6-2832-44ba-ab2a-23672025a1e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:19.484881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0805d6-2832-44ba-ab2a-23672025a1e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:19.486600 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0805d6-2832-44ba-ab2a-23672025a1e5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:19.506047 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 407/807] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:19.507019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0805d6-2832-44ba-ab2a-23672025a1e5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:19.575642 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567a580-bbfe-42ab-b8d0-6682863cc175 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:19.598478 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:19.623333 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ffca4-f6f1-489f-96ac-a219ed236153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:19.624646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ffca4-f6f1-489f-96ac-a219ed236153 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:19.625114 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ffca4-f6f1-489f-96ac-a219ed236153 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:19.626566 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ffca4-f6f1-489f-96ac-a219ed236153 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:19.627528 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ffca4-f6f1-489f-96ac-a219ed236153 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:19.628017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ffca4-f6f1-489f-96ac-a219ed236153 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:19.629634 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ffca4-f6f1-489f-96ac-a219ed236153 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:19.641329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 401/808] 10.222.0.65 () {56 vars in 1124 bytes} [Thu May 19 06:17:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:19.642620 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ffca4-f6f1-489f-96ac-a219ed236153 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:19.689391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0805d6-2832-44ba-ab2a-23672025a1e5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:19.765046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47185cbd-0bd1-44db-ae2e-8220e8534231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:19.766276 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47185cbd-0bd1-44db-ae2e-8220e8534231 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:19.767108 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47185cbd-0bd1-44db-ae2e-8220e8534231 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:19.767658 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47185cbd-0bd1-44db-ae2e-8220e8534231 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:19.769737 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47185cbd-0bd1-44db-ae2e-8220e8534231 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:19.770407 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47185cbd-0bd1-44db-ae2e-8220e8534231 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:19.772884 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47185cbd-0bd1-44db-ae2e-8220e8534231 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:19.791008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 408/809] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:19.792531 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47185cbd-0bd1-44db-ae2e-8220e8534231 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:19.821733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:19.824240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ffca4-f6f1-489f-96ac-a219ed236153 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:19.875048 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3723a39-d53c-408c-92a1-381b42ea7750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:19.875768 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3723a39-d53c-408c-92a1-381b42ea7750 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:19.876060 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3723a39-d53c-408c-92a1-381b42ea7750 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:19.876341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3723a39-d53c-408c-92a1-381b42ea7750 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:19.887355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3723a39-d53c-408c-92a1-381b42ea7750 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:19.889113 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3723a39-d53c-408c-92a1-381b42ea7750 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:19.911556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-b3723a39-d53c-408c-92a1-381b42ea7750 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 06:17:19.911556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:19.911556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:19.911556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:19.911556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:19.911556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:19.911556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:19.911556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:19.911556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:19.911556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:19.911556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:19.911556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:19.911556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:17:19.911556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:17:19.911556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:17:19.911556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:19.913428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:19.914654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:19.914654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:19.914654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:17:19.914654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:17:19.914654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:17:19.914654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:17:19.914654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:17:19.914654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:17:19.914654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 06:17:19.914654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:19.914654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 402/810] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 06:17:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:19.914654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3723a39-d53c-408c-92a1-381b42ea7750 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:19.960971 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47185cbd-0bd1-44db-ae2e-8220e8534231 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:20.007592 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:20.010894 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509bd2a-3ad0-4cd2-8500-de86192a27b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:20.011857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509bd2a-3ad0-4cd2-8500-de86192a27b1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:20.012154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509bd2a-3ad0-4cd2-8500-de86192a27b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:20.012519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509bd2a-3ad0-4cd2-8500-de86192a27b1 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:20.013808 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509bd2a-3ad0-4cd2-8500-de86192a27b1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:20.014705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509bd2a-3ad0-4cd2-8500-de86192a27b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:20.017658 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509bd2a-3ad0-4cd2-8500-de86192a27b1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:20.030102 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 409/811] 10.222.0.65 () {56 vars in 1124 bytes} [Thu May 19 06:17:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:20.030102 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509bd2a-3ad0-4cd2-8500-de86192a27b1 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:20.121763 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3723a39-d53c-408c-92a1-381b42ea7750 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:20.143060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:20.170434 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509bd2a-3ad0-4cd2-8500-de86192a27b1 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:20.202626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43cd32-b599-4c03-995e-13e1ae459e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:20.203576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43cd32-b599-4c03-995e-13e1ae459e7a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:20.203866 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43cd32-b599-4c03-995e-13e1ae459e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:20.204143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43cd32-b599-4c03-995e-13e1ae459e7a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:20.213414 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43cd32-b599-4c03-995e-13e1ae459e7a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:20.213776 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15767a-1af2-451b-b7ba-615eb43d5e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:20.214900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15767a-1af2-451b-b7ba-615eb43d5e4b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:20.215124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43cd32-b599-4c03-995e-13e1ae459e7a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:20.215836 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15767a-1af2-451b-b7ba-615eb43d5e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:20.216418 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15767a-1af2-451b-b7ba-615eb43d5e4b None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:20.217930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15767a-1af2-451b-b7ba-615eb43d5e4b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:20.218540 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15767a-1af2-451b-b7ba-615eb43d5e4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:20.221142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15767a-1af2-451b-b7ba-615eb43d5e4b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:20.232922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-5e43cd32-b599-4c03-995e-13e1ae459e7a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 06:17:20.232922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:20.232922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:20.232922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:20.232922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:20.232922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:20.232922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:20.232922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:20.232922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:20.232922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:20.232922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:20.232922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:20.232922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:17:20.232922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:17:20.232922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:17:20.232922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:20.234544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:20.235705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:20.235705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:20.235705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:17:20.235705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:17:20.235705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:17:20.235705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:17:20.235705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:17:20.235705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:17:20.235705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 06:17:20.235705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:20.235705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 403/812] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 06:17:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:20.235705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43cd32-b599-4c03-995e-13e1ae459e7a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:20.255784 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 410/813] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 06:17:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:20.257862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15767a-1af2-451b-b7ba-615eb43d5e4b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:20.462012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15767a-1af2-451b-b7ba-615eb43d5e4b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:20.495656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43cd32-b599-4c03-995e-13e1ae459e7a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:20.497479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:20.521566 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fa078-03cd-4667-a633-afe777d74031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:20.522290 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fa078-03cd-4667-a633-afe777d74031 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:20.523088 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fa078-03cd-4667-a633-afe777d74031 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:20.523088 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fa078-03cd-4667-a633-afe777d74031 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:20.523946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fa078-03cd-4667-a633-afe777d74031 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:20.524447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fa078-03cd-4667-a633-afe777d74031 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:20.526833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fa078-03cd-4667-a633-afe777d74031 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:20.539012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:20.541120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d5a5f-75ad-46bc-8637-1c2839f0ece5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:20.542977 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d5a5f-75ad-46bc-8637-1c2839f0ece5 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:20.543779 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d5a5f-75ad-46bc-8637-1c2839f0ece5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:20.544408 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d5a5f-75ad-46bc-8637-1c2839f0ece5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:20.545806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d5a5f-75ad-46bc-8637-1c2839f0ece5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:20.547500 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d5a5f-75ad-46bc-8637-1c2839f0ece5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:20.560256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 411/814] 10.222.0.65 () {56 vars in 1149 bytes} [Thu May 19 06:17:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:20.561896 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fa078-03cd-4667-a633-afe777d74031 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:20.575023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 404/815] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 06:17:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:17:20.576268 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d5a5f-75ad-46bc-8637-1c2839f0ece5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:20.710535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d5a5f-75ad-46bc-8637-1c2839f0ece5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:20.756154 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f51ba-47a6-4695-b740-11a17eef0440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:20.757412 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f51ba-47a6-4695-b740-11a17eef0440 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:20.757919 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f51ba-47a6-4695-b740-11a17eef0440 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:20.758401 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f51ba-47a6-4695-b740-11a17eef0440 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:20.771301 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f51ba-47a6-4695-b740-11a17eef0440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:20.773223 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f51ba-47a6-4695-b740-11a17eef0440 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:20.795056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-a42f51ba-47a6-4695-b740-11a17eef0440 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:20.795056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:20.795056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:20.795056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:20.795056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:20.795056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:20.795056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:20.795056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:20.795056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:20.795056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:20.795056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:20.795056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:20.795056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:20.795056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:20.795056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:20.796038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:20.796038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:20.796038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:20.796038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:20.796038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:20.796038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:20.796038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:20.796038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:20.796038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:20.796038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:20.796038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:20.796038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:20.796038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:20.796038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:20.796038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:20.796038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:20.796038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:20.796932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:20.796932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:20.796932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:20.796932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:20.799555 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 405/816] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:20.812643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f51ba-47a6-4695-b740-11a17eef0440 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:20.828085 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:20.898228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fa078-03cd-4667-a633-afe777d74031 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:20.942283 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f51ba-47a6-4695-b740-11a17eef0440 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:20.965993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e0fe2-7926-43d4-924e-f31a8bfa7a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:20.967254 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e0fe2-7926-43d4-924e-f31a8bfa7a58 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:20.967794 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e0fe2-7926-43d4-924e-f31a8bfa7a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:20.968169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e0fe2-7926-43d4-924e-f31a8bfa7a58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:20.990629 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69b137-a1a9-4820-855d-09d01ac5d87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:20.991318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69b137-a1a9-4820-855d-09d01ac5d87f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:20.991880 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69b137-a1a9-4820-855d-09d01ac5d87f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:20.992161 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69b137-a1a9-4820-855d-09d01ac5d87f None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:20.993498 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69b137-a1a9-4820-855d-09d01ac5d87f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:20.993813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69b137-a1a9-4820-855d-09d01ac5d87f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:20.996600 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69b137-a1a9-4820-855d-09d01ac5d87f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:21.015539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 406/817] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:21.020881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69b137-a1a9-4820-855d-09d01ac5d87f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:21.052475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:21.061020 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e0fe2-7926-43d4-924e-f31a8bfa7a58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:21.062797 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e0fe2-7926-43d4-924e-f31a8bfa7a58 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:21.095769 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:21.140386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69b137-a1a9-4820-855d-09d01ac5d87f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:21.191241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 412/818] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:17:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:17:21.192741 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0587953-9070-4ad4-8fba-326f76a78c58 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:21.193030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0587953-9070-4ad4-8fba-326f76a78c58 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:21.193331 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0587953-9070-4ad4-8fba-326f76a78c58 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:21.196699 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6cb380-a7de-4683-abf8-ce10fe843732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:21.197637 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cb380-a7de-4683-abf8-ce10fe843732 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:21.197980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cb380-a7de-4683-abf8-ce10fe843732 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:21.198451 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cb380-a7de-4683-abf8-ce10fe843732 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:21.199015 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 413/819] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:21.199928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0587953-9070-4ad4-8fba-326f76a78c58 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:21.210257 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6cb380-a7de-4683-abf8-ce10fe843732 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:21.212655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6cb380-a7de-4683-abf8-ce10fe843732 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:21.236744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-ac6cb380-a7de-4683-abf8-ce10fe843732 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:21.236744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:21.236744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:21.236744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:21.236744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:21.236744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:21.236744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:21.236744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:21.236744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:21.236744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:21.236744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:21.236744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:21.236744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:21.236744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:21.236744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:21.237910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:21.237910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:21.237910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:21.237910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:21.237910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:21.237910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:21.237910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:21.237910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:21.237910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:21.237910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:21.237910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:21.237910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:21.237910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:21.237910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:21.237910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:21.237910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:21.237910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:21.238872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:21.238872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:21.238872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:21.238872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:21.238872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:21.238872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:21.238872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:21.238872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:21.239836 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 407/820] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:21.241601 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413e806-58f3-470b-a4d7-7efe6032c76c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:21.241960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413e806-58f3-470b-a4d7-7efe6032c76c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:21.242300 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413e806-58f3-470b-a4d7-7efe6032c76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:21.264743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4413e806-58f3-470b-a4d7-7efe6032c76c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:21.266201 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4413e806-58f3-470b-a4d7-7efe6032c76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:21.323252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:21.328319 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0587953-9070-4ad4-8fba-326f76a78c58 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:21.389343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411dc1d-2033-480f-828a-daa3dfecc32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:21.390694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411dc1d-2033-480f-828a-daa3dfecc32b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:21.391156 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411dc1d-2033-480f-828a-daa3dfecc32b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:21.391628 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411dc1d-2033-480f-828a-daa3dfecc32b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:21.412014 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411dc1d-2033-480f-828a-daa3dfecc32b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:21.414681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411dc1d-2033-480f-828a-daa3dfecc32b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:21.430958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 408/821] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:21.433445 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413e806-58f3-470b-a4d7-7efe6032c76c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:21.478040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-2411dc1d-2033-480f-828a-daa3dfecc32b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:21.478040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:21.478040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:21.478040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:21.478040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:21.478040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:21.478040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:21.478040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:21.478040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:21.478040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:21.478040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:21.478040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:21.478040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:21.478040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:21.478040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:21.479607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:21.479607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:21.479607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:21.479607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:21.479607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:21.479607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:21.479607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:21.479607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:21.479607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:21.479607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:21.479607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:21.479607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:21.479607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:21.479607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:21.479607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:21.479607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:21.479607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:21.480491 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:21.480491 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:21.480491 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:21.480491 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:21.480933 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 414/822] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:21.482516 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411dc1d-2033-480f-828a-daa3dfecc32b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:21.518808 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:21.588277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413e806-58f3-470b-a4d7-7efe6032c76c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:21.621586 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411dc1d-2033-480f-828a-daa3dfecc32b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:21.650185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e591ec-846b-4880-bd13-3d7a20a01d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:21.651345 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e591ec-846b-4880-bd13-3d7a20a01d09 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:21.651840 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e591ec-846b-4880-bd13-3d7a20a01d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:21.652391 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e591ec-846b-4880-bd13-3d7a20a01d09 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:21.654278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e591ec-846b-4880-bd13-3d7a20a01d09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:21.654742 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e591ec-846b-4880-bd13-3d7a20a01d09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:21.657576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e591ec-846b-4880-bd13-3d7a20a01d09 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:21.663166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e08e0-b0f9-4b9d-a3e4-b9e14ce0462a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:21.663987 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e08e0-b0f9-4b9d-a3e4-b9e14ce0462a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:21.664270 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e08e0-b0f9-4b9d-a3e4-b9e14ce0462a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:21.664555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e08e0-b0f9-4b9d-a3e4-b9e14ce0462a None admin] PATH_INFO: `/v3/services/network` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:21.665225 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4e08e0-b0f9-4b9d-a3e4-b9e14ce0462a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:21.667345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4e08e0-b0f9-4b9d-a3e4-b9e14ce0462a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:21.679745 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 409/823] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:17:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:21.681759 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e591ec-846b-4880-bd13-3d7a20a01d09 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:21.689843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-8d4e08e0-b0f9-4b9d-a3e4-b9e14ce0462a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 06:17:21.689843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:21.689843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:21.689843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:21.689843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:21.689843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:21.689843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:21.689843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:21.689843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 06:17:21.689843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:21.689843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:21.689843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:21.689843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:21.689843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:21.689843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:21.689843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:21.689843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:21.690887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:21.690887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:21.690887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:21.690887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:21.690887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:21.690887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:21.690887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:21.690887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:21.690887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:21.690887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:21.690887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:21.690887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:21.690887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:21.690887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:21.690887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:21.690887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:21.691746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:21.691746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:21.691746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:21.691746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:21.691746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:21.691746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:21.691746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:21.691746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:21.691746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:21.691746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:21.691746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:21.691746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:21.691746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 06:17:21.691746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:21.692483 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 415/824] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:21 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:21.698838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f587d8f-a3e6-4167-8b6d-9fdf9649d4ba None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:21.698838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f587d8f-a3e6-4167-8b6d-9fdf9649d4ba None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:21.698838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f587d8f-a3e6-4167-8b6d-9fdf9649d4ba None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:21.699825 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 416/825] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:21.700790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f587d8f-a3e6-4167-8b6d-9fdf9649d4ba None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:21.736010 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:21.853019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f587d8f-a3e6-4167-8b6d-9fdf9649d4ba None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:21.854753 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e591ec-846b-4880-bd13-3d7a20a01d09 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:21.880153 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:21.897238 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22871cd6-6508-4576-adac-4a0911ea7d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:21.898180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22871cd6-6508-4576-adac-4a0911ea7d9c None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:21.898487 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22871cd6-6508-4576-adac-4a0911ea7d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:21.898737 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22871cd6-6508-4576-adac-4a0911ea7d9c None admin] PATH_INFO: `/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/aee77a3ed91445ab8b7f58e7f8bc348a/roles/5cc9ca1bc48b4d749d21d7111b14e187` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:21.907567 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f8c3d-fd80-4ddc-8931-0fad3edbd4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:21.908487 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f8c3d-fd80-4ddc-8931-0fad3edbd4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:21.908985 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f8c3d-fd80-4ddc-8931-0fad3edbd4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:21.909454 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f8c3d-fd80-4ddc-8931-0fad3edbd4d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:21.910444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f8c3d-fd80-4ddc-8931-0fad3edbd4d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:21.910949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f8c3d-fd80-4ddc-8931-0fad3edbd4d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:21.913155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f8c3d-fd80-4ddc-8931-0fad3edbd4d4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:21.918912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22871cd6-6508-4576-adac-4a0911ea7d9c None admin] RBAC: Authorizing `identity:create_grant(project_id=a3d399dc95424b848b736c0d0298d9a9, user_id=aee77a3ed91445ab8b7f58e7f8bc348a, role_id=5cc9ca1bc48b4d749d21d7111b14e187)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:21.920376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22871cd6-6508-4576-adac-4a0911ea7d9c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:21.929001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 410/826] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:21.931246 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f8c3d-fd80-4ddc-8931-0fad3edbd4d4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:21.944143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 417/827] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:17:21 2022] PUT /identity/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/aee77a3ed91445ab8b7f58e7f8bc348a/roles/5cc9ca1bc48b4d749d21d7111b14e187 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:17:21.946716 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ae420-bcb4-4d18-bf26-5d9baad1b228 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:21.948918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ae420-bcb4-4d18-bf26-5d9baad1b228 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:21.949543 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ae420-bcb4-4d18-bf26-5d9baad1b228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:21.985851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3f4ae420-bcb4-4d18-bf26-5d9baad1b228 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:21.987617 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3f4ae420-bcb4-4d18-bf26-5d9baad1b228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:22.094220 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f8c3d-fd80-4ddc-8931-0fad3edbd4d4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:22.124572 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:22.154805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 418/828] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:22.156397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ae420-bcb4-4d18-bf26-5d9baad1b228 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:22.161226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf43a5-e0f6-4c0a-8ea6-df4693ff85e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:22.162510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf43a5-e0f6-4c0a-8ea6-df4693ff85e4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:22.163043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf43a5-e0f6-4c0a-8ea6-df4693ff85e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:22.163719 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf43a5-e0f6-4c0a-8ea6-df4693ff85e4 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:22.164959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf43a5-e0f6-4c0a-8ea6-df4693ff85e4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:22.165697 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf43a5-e0f6-4c0a-8ea6-df4693ff85e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:22.169594 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf43a5-e0f6-4c0a-8ea6-df4693ff85e4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:22.177772 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 411/829] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 06:17:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:22.185234 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf43a5-e0f6-4c0a-8ea6-df4693ff85e4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:22.252279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:22.333552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ae420-bcb4-4d18-bf26-5d9baad1b228 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:22.365488 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf43a5-e0f6-4c0a-8ea6-df4693ff85e4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:22.392368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbef22-e1c6-4f82-8ec8-fbe5826b9474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:22.395144 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbef22-e1c6-4f82-8ec8-fbe5826b9474 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:22.396065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbef22-e1c6-4f82-8ec8-fbe5826b9474 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:22.397105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbef22-e1c6-4f82-8ec8-fbe5826b9474 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:22.414706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9166c10-5db6-441d-b5b3-bdefcab5f804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:22.415653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debbef22-e1c6-4f82-8ec8-fbe5826b9474 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:22.415823 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9166c10-5db6-441d-b5b3-bdefcab5f804 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:22.416737 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9166c10-5db6-441d-b5b3-bdefcab5f804 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:22.417015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9166c10-5db6-441d-b5b3-bdefcab5f804 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:22.418067 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9166c10-5db6-441d-b5b3-bdefcab5f804 None admin] RBAC: Adding query filter params (type=network) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:22.418485 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9166c10-5db6-441d-b5b3-bdefcab5f804 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:22.420373 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9166c10-5db6-441d-b5b3-bdefcab5f804 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:22.420727 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debbef22-e1c6-4f82-8ec8-fbe5826b9474 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:22.427591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 412/830] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 06:17:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:22.428794 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9166c10-5db6-441d-b5b3-bdefcab5f804 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:22.454788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-debbef22-e1c6-4f82-8ec8-fbe5826b9474 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:22.454788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:22.454788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:22.454788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:22.454788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:22.454788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:22.454788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:22.454788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:22.454788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:22.454788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:22.454788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:22.454788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:22.454788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:22.454788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:22.454788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:22.456004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:22.456004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:22.456004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:22.456004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:22.456004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:22.456004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:22.456004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:22.456004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:22.456004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:22.456004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:22.456004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:22.456004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:22.456004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:22.456004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:22.456004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:22.456004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:22.456004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:22.457554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:22.457554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:22.457554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:22.457554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:22.457554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:22.457554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:22.457554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:22.457554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:22.466065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 419/831] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:22.466065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbef22-e1c6-4f82-8ec8-fbe5826b9474 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:22.564186 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:22.581672 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9166c10-5db6-441d-b5b3-bdefcab5f804 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:22.584534 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:22.623626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591a4c8-9bc8-45ce-a203-41c40ef676fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:22.624455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591a4c8-9bc8-45ce-a203-41c40ef676fc None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:22.624747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591a4c8-9bc8-45ce-a203-41c40ef676fc None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:22.625027 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591a4c8-9bc8-45ce-a203-41c40ef676fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:22.640166 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591a4c8-9bc8-45ce-a203-41c40ef676fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:17:22.645006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbef22-e1c6-4f82-8ec8-fbe5826b9474 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:22.650183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-0591a4c8-9bc8-45ce-a203-41c40ef676fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:17:22.650183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:22.650183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:22.650183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:22.650183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:22.650183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:22.650183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:22.650183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:22.650183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:22.650183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:22.650183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:22.650183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:22.650183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:17:22.650183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:17:22.650183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:17:22.650183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:17:22.651799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:17:22.651799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:17:22.651799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:17:22.651799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:17:22.651799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:22.651799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:22.651799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:22.651799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:22.651799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:22.651799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:22.651799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:22.651799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:22.651799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:22.651799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:22.651799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:22.651799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:22.652892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:22.652892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:22.652892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:22.652892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:22.652892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:17:22.652892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:17:22.652892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:17:22.652892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:17:22.652892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:17:22.652892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:17:22.652892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:17:22.652892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:22.661080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 413/832] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:17:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:22.662360 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591a4c8-9bc8-45ce-a203-41c40ef676fc None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:22.701025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f836540-5679-43da-b080-681d82299956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:22.702343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f836540-5679-43da-b080-681d82299956 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:22.702891 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f836540-5679-43da-b080-681d82299956 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:22.703394 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f836540-5679-43da-b080-681d82299956 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:22.704426 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f836540-5679-43da-b080-681d82299956 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:22.706326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f836540-5679-43da-b080-681d82299956 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:22.733697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 420/833] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 06:17:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:17:22.735113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f836540-5679-43da-b080-681d82299956 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:22.885857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f836540-5679-43da-b080-681d82299956 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:22.889450 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:22.896503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591a4c8-9bc8-45ce-a203-41c40ef676fc None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:22.940286 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a507da-657b-4f02-8dcd-82b82fb5ca8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:22.941444 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a507da-657b-4f02-8dcd-82b82fb5ca8b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:22.941959 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a507da-657b-4f02-8dcd-82b82fb5ca8b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:22.942492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a507da-657b-4f02-8dcd-82b82fb5ca8b None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:22.943542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a507da-657b-4f02-8dcd-82b82fb5ca8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:22.944084 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a507da-657b-4f02-8dcd-82b82fb5ca8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:22.945977 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a507da-657b-4f02-8dcd-82b82fb5ca8b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:22.958715 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 421/834] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:17:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:22.966685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a507da-657b-4f02-8dcd-82b82fb5ca8b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:22.995964 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7eea2-6add-42a9-93d8-86e59f871546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:22.996661 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7eea2-6add-42a9-93d8-86e59f871546 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:22.996946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7eea2-6add-42a9-93d8-86e59f871546 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:22.997224 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7eea2-6add-42a9-93d8-86e59f871546 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:22.998061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7eea2-6add-42a9-93d8-86e59f871546 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:22.998371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7eea2-6add-42a9-93d8-86e59f871546 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:22.999873 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7eea2-6add-42a9-93d8-86e59f871546 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:23.023750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 414/835] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:17:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:23.033231 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7eea2-6add-42a9-93d8-86e59f871546 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:23.073089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:23.119154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a507da-657b-4f02-8dcd-82b82fb5ca8b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:23.167937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624e3f5-abff-4be3-bef9-aa3772ad2338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:23.168701 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624e3f5-abff-4be3-bef9-aa3772ad2338 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:23.168991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624e3f5-abff-4be3-bef9-aa3772ad2338 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:23.169272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624e3f5-abff-4be3-bef9-aa3772ad2338 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:23.189572 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624e3f5-abff-4be3-bef9-aa3772ad2338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:23.190844 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624e3f5-abff-4be3-bef9-aa3772ad2338 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:23.193126 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7eea2-6add-42a9-93d8-86e59f871546 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:23.210731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-2624e3f5-abff-4be3-bef9-aa3772ad2338 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:23.210731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:23.210731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:23.210731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:23.210731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:23.210731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:23.210731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:23.210731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:23.210731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:23.210731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:23.210731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:23.210731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:23.210731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:23.210731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:23.210731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:23.211921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:23.211921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:23.211921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:23.211921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:23.211921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:23.211921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:23.211921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:23.211921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:23.211921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:23.211921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:23.211921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:23.211921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:23.211921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:23.211921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:23.211921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:23.211921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:23.211921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:23.213064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:23.213064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:23.213064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:23.213064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:23.215028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 422/836] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:23.220583 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624e3f5-abff-4be3-bef9-aa3772ad2338 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:23.245092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ca92f-730e-42ab-b2e9-3d5b3028e067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:23.246221 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ca92f-730e-42ab-b2e9-3d5b3028e067 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:23.246669 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ca92f-730e-42ab-b2e9-3d5b3028e067 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:23.247115 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ca92f-730e-42ab-b2e9-3d5b3028e067 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:23.248107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ca92f-730e-42ab-b2e9-3d5b3028e067 None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=2a09f4878eac4d60a9a7dcedba15c05d) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:23.248621 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ca92f-730e-42ab-b2e9-3d5b3028e067 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:23.250300 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ca92f-730e-42ab-b2e9-3d5b3028e067 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:23.253710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:23.265728 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 415/837] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:17:23 2022] GET /identity/v3/role_assignments?user.id=2a09f4878eac4d60a9a7dcedba15c05d&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:23.348103 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624e3f5-abff-4be3-bef9-aa3772ad2338 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:23.383040 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc24b09-403f-4d5c-99c8-4ec6fd183b3b None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:23.383643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc24b09-403f-4d5c-99c8-4ec6fd183b3b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:23.384136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc24b09-403f-4d5c-99c8-4ec6fd183b3b None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:23.386273 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 416/838] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:23.390529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b95876-8f63-49fa-8d03-7140ac4b9608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:23.391224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b95876-8f63-49fa-8d03-7140ac4b9608 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:23.391541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b95876-8f63-49fa-8d03-7140ac4b9608 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:23.391826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b95876-8f63-49fa-8d03-7140ac4b9608 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:23.392663 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b95876-8f63-49fa-8d03-7140ac4b9608 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:23.393054 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b95876-8f63-49fa-8d03-7140ac4b9608 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:23.394708 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b95876-8f63-49fa-8d03-7140ac4b9608 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:23.395653 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9b556-33e9-4584-8d0e-8b842a1c71b8 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:23.395948 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9b556-33e9-4584-8d0e-8b842a1c71b8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:23.396219 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9b556-33e9-4584-8d0e-8b842a1c71b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:23.410449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 423/839] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:23.416146 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b95876-8f63-49fa-8d03-7140ac4b9608 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:23.419360 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-66b9b556-33e9-4584-8d0e-8b842a1c71b8 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:23.420766 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-66b9b556-33e9-4584-8d0e-8b842a1c71b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:23.523914 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:23.529425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 417/840] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:23.553092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9b556-33e9-4584-8d0e-8b842a1c71b8 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:23.566112 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b95876-8f63-49fa-8d03-7140ac4b9608 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:23.578237 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:23.613055 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd13a3-096a-4f88-a0f5-734acd412f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:23.613930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd13a3-096a-4f88-a0f5-734acd412f6f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:23.614219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd13a3-096a-4f88-a0f5-734acd412f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:23.614501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd13a3-096a-4f88-a0f5-734acd412f6f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:23.623308 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd13a3-096a-4f88-a0f5-734acd412f6f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:23.624786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd13a3-096a-4f88-a0f5-734acd412f6f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:23.643148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-7cdd13a3-096a-4f88-a0f5-734acd412f6f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 06:17:23.643148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:23.643148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:23.643148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:23.643148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:23.643148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:23.643148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:23.643148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:23.643148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:23.643148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:23.643148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:23.643148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:23.643148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:17:23.643148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:17:23.643148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:17:23.643148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:23.644198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:23.645733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:23.645733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:23.645733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:17:23.645733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:17:23.645733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:17:23.645733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:17:23.645733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:17:23.645733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:17:23.645733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 06:17:23.645733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:23.645733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 424/841] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:17:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:23.657914 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd13a3-096a-4f88-a0f5-734acd412f6f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:23.704311 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9b556-33e9-4584-8d0e-8b842a1c71b8 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:23.775470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648fb1f-b68e-4b97-8e24-9adeaf572b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:23.778455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648fb1f-b68e-4b97-8e24-9adeaf572b22 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:23.779183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648fb1f-b68e-4b97-8e24-9adeaf572b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:23.779671 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648fb1f-b68e-4b97-8e24-9adeaf572b22 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:23.781400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648fb1f-b68e-4b97-8e24-9adeaf572b22 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:23.784149 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648fb1f-b68e-4b97-8e24-9adeaf572b22 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:23.813584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-d648fb1f-b68e-4b97-8e24-9adeaf572b22 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 06:17:23.813584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:23.813584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:23.813584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:23.813584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:23.813584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:23.813584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:23.813584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:23.813584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 06:17:23.813584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:23.813584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:23.813584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:23.813584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:23.813584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:23.813584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:23.813584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:23.813584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:23.815639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:23.815639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:23.815639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:23.815639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:23.815639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:23.815639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:23.815639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:23.815639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:23.815639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:23.815639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:23.815639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:23.815639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:23.815639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:23.815639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:23.815639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:23.815639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:23.817305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:23.817305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:23.817305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:23.817305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:23.817305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:23.817305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:23.817305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:23.817305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:23.817305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:23.817305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:23.817305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:23.817305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:23.817305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 06:17:23.817305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:23.821794 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 418/842] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:17:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:23.826386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648fb1f-b68e-4b97-8e24-9adeaf572b22 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:23.874094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd13a3-096a-4f88-a0f5-734acd412f6f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:23.941520 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6809c-4b56-42e3-b1bc-5644b9ccfc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:23.942601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6809c-4b56-42e3-b1bc-5644b9ccfc85 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:23.943085 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6809c-4b56-42e3-b1bc-5644b9ccfc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:23.943744 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6809c-4b56-42e3-b1bc-5644b9ccfc85 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:23.945470 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d6809c-4b56-42e3-b1bc-5644b9ccfc85 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:23.946009 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d6809c-4b56-42e3-b1bc-5644b9ccfc85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:23.949584 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d6809c-4b56-42e3-b1bc-5644b9ccfc85 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:23.968160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:23.989668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 425/843] 10.222.0.65 () {56 vars in 1135 bytes} [Thu May 19 06:17:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:23.997404 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6809c-4b56-42e3-b1bc-5644b9ccfc85 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:24.004594 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:24.039302 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648fb1f-b68e-4b97-8e24-9adeaf572b22 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:24.110855 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63ce6d-49de-41d3-8c84-206cfc1e2a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:24.112197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63ce6d-49de-41d3-8c84-206cfc1e2a7c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:24.112701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63ce6d-49de-41d3-8c84-206cfc1e2a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:24.113168 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63ce6d-49de-41d3-8c84-206cfc1e2a7c None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:24.114941 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63ce6d-49de-41d3-8c84-206cfc1e2a7c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:24.115483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63ce6d-49de-41d3-8c84-206cfc1e2a7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:24.118291 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63ce6d-49de-41d3-8c84-206cfc1e2a7c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:24.134132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 419/844] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:17:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:24.136356 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63ce6d-49de-41d3-8c84-206cfc1e2a7c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:24.206218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6809c-4b56-42e3-b1bc-5644b9ccfc85 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:24.279342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a56ad-7507-4540-8e52-724bc225ec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:24.281110 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a56ad-7507-4540-8e52-724bc225ec32 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:24.281906 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a56ad-7507-4540-8e52-724bc225ec32 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:24.283162 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a56ad-7507-4540-8e52-724bc225ec32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:24.313348 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63ce6d-49de-41d3-8c84-206cfc1e2a7c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:24.364237 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86846f19-d5a4-40bb-880b-d995b95c90a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:24.364913 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86846f19-d5a4-40bb-880b-d995b95c90a1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:24.365198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86846f19-d5a4-40bb-880b-d995b95c90a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:24.365519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86846f19-d5a4-40bb-880b-d995b95c90a1 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:24.366223 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86846f19-d5a4-40bb-880b-d995b95c90a1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:24.366532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86846f19-d5a4-40bb-880b-d995b95c90a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:24.367963 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86846f19-d5a4-40bb-880b-d995b95c90a1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:24.374193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 420/845] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:17:24 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:24.379565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcc4fc-9017-4f3d-b363-165b28ea2153 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:24.379930 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcc4fc-9017-4f3d-b363-165b28ea2153 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:24.380318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcc4fc-9017-4f3d-b363-165b28ea2153 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:24.386868 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 421/846] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:24.389069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f31e4-2341-416c-91ac-5a7bedba7752 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:24.389396 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f31e4-2341-416c-91ac-5a7bedba7752 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:24.389682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f31e4-2341-416c-91ac-5a7bedba7752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:24.398965 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:24.410636 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a56ad-7507-4540-8e52-724bc225ec32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:24.412930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a56ad-7507-4540-8e52-724bc225ec32 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:24.420573 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9f6f31e4-2341-416c-91ac-5a7bedba7752 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:24.422218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9f6f31e4-2341-416c-91ac-5a7bedba7752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:24.427077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:24.549177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 422/847] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:24.584119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f31e4-2341-416c-91ac-5a7bedba7752 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:24.592839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 426/848] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:17:23 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:17:24.598665 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a56ad-7507-4540-8e52-724bc225ec32 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:24.756185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f31e4-2341-416c-91ac-5a7bedba7752 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:24.774101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:24.784288 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a56ad-7507-4540-8e52-724bc225ec32 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:24.825872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f211a-9f74-493b-a5a5-3ddf273584c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:24.827420 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f211a-9f74-493b-a5a5-3ddf273584c1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:24.827954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f211a-9f74-493b-a5a5-3ddf273584c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:24.828083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ce118-c2ac-44f8-9c39-1d340b81f5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:24.828466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f211a-9f74-493b-a5a5-3ddf273584c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:24.829347 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ce118-c2ac-44f8-9c39-1d340b81f5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:24.829867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ce118-c2ac-44f8-9c39-1d340b81f5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:24.830579 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ce118-c2ac-44f8-9c39-1d340b81f5b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:24.844618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ce118-c2ac-44f8-9c39-1d340b81f5b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:24.846833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ce118-c2ac-44f8-9c39-1d340b81f5b6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:24.849179 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f211a-9f74-493b-a5a5-3ddf273584c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:17:24.858230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:24.868946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-572f211a-9f74-493b-a5a5-3ddf273584c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:17:24.868946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:24.868946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:24.868946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:24.868946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:24.868946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:24.868946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:24.868946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:24.868946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:24.868946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:24.868946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:24.868946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:24.868946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:17:24.868946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:17:24.868946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:17:24.868946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:17:24.871212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:17:24.871212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:17:24.871212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:17:24.871212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:17:24.871212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:24.871212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:24.871212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:24.871212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:24.871212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:24.871212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:24.871212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:24.871212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:24.871212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:24.871212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:24.871212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:24.871212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:24.872245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:24.872245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:24.872245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:24.872245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:24.872245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:17:24.872245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:17:24.872245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:17:24.872245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:17:24.872245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:17:24.872245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:17:24.872245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:17:24.872245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:24.873229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-2b5ce118-c2ac-44f8-9c39-1d340b81f5b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:24.873229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:24.873229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:24.873229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:24.873229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:24.873229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:24.873229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:24.873229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:24.873229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:24.873229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:24.873229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:24.873229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:24.873229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:24.873229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:24.873229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:24.875976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:24.875976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:24.875976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:24.875976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:24.875976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:24.875976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:24.875976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:24.875976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:24.875976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:24.875976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:24.875976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:24.875976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:24.875976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:24.875976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:24.875976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:24.875976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:24.875976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:24.879010 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:24.879010 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:24.879010 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:24.879010 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:24.879010 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 427/849] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:24.879010 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ce118-c2ac-44f8-9c39-1d340b81f5b6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:24.891099 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 423/850] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:17:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:24.894013 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f211a-9f74-493b-a5a5-3ddf273584c1 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:25.017826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ce118-c2ac-44f8-9c39-1d340b81f5b6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:25.061259 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f211a-9f74-493b-a5a5-3ddf273584c1 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:25.085275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7fd60-b222-44dc-941b-da93e584cb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:25.086513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7fd60-b222-44dc-941b-da93e584cb41 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:25.087029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7fd60-b222-44dc-941b-da93e584cb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:25.087549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7fd60-b222-44dc-941b-da93e584cb41 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:25.089337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7fd60-b222-44dc-941b-da93e584cb41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:25.089903 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7fd60-b222-44dc-941b-da93e584cb41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:25.092485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7fd60-b222-44dc-941b-da93e584cb41 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:25.107770 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5238e1-3750-46a9-a964-99adc4f02eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:25.108824 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5238e1-3750-46a9-a964-99adc4f02eba None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:25.110010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5238e1-3750-46a9-a964-99adc4f02eba None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:25.110544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5238e1-3750-46a9-a964-99adc4f02eba None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:25.111835 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5238e1-3750-46a9-a964-99adc4f02eba None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:25.112383 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5238e1-3750-46a9-a964-99adc4f02eba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:25.114019 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5238e1-3750-46a9-a964-99adc4f02eba None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:25.115521 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 428/851] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:25.129332 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 424/852] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:17:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:25.129572 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7fd60-b222-44dc-941b-da93e584cb41 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:25.131545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5238e1-3750-46a9-a964-99adc4f02eba None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:25.151421 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:25.217779 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:25.279445 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5238e1-3750-46a9-a964-99adc4f02eba None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:25.324096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7fd60-b222-44dc-941b-da93e584cb41 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:25.344527 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81d468-9479-43a2-beeb-81edbc834924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:25.346073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81d468-9479-43a2-beeb-81edbc834924 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:25.346570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81d468-9479-43a2-beeb-81edbc834924 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:25.347088 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81d468-9479-43a2-beeb-81edbc834924 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:25.359433 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81d468-9479-43a2-beeb-81edbc834924 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:25.361326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81d468-9479-43a2-beeb-81edbc834924 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:25.374469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ab361-73eb-4934-8d8f-39d0781f4a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:25.375286 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ab361-73eb-4934-8d8f-39d0781f4a16 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:25.375576 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ab361-73eb-4934-8d8f-39d0781f4a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:25.375861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ab361-73eb-4934-8d8f-39d0781f4a16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:25.381701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-0c81d468-9479-43a2-beeb-81edbc834924 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:25.381701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:25.381701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:25.381701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:25.381701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:25.381701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:25.381701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:25.381701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:25.381701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:25.381701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:25.381701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:25.381701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:25.381701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:25.381701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:25.381701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:25.382619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:25.382619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:25.382619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:25.382619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:25.382619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:25.382619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:25.382619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:25.382619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:25.382619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:25.382619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:25.382619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:25.382619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:25.382619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:25.382619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:25.382619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:25.382619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:25.382619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:25.383529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:25.383529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:25.383529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:25.383529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:25.384192 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ab361-73eb-4934-8d8f-39d0781f4a16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:25.390882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ab361-73eb-4934-8d8f-39d0781f4a16 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:25.391081 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 425/853] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:25.392252 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81d468-9479-43a2-beeb-81edbc834924 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:25.418350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-d87ab361-73eb-4934-8d8f-39d0781f4a16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:25.418350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:25.418350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:25.418350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:25.418350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:25.418350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:25.418350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:25.418350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:25.418350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:25.418350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:25.418350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:25.418350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:25.418350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:25.418350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:25.418350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:25.419349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:25.419349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:25.419349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:25.419349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:25.419349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:25.419349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:25.419349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:25.419349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:25.419349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:25.419349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:25.419349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:25.419349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:25.419349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:25.419349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:25.419349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:25.419349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:25.419349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:25.420257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:25.420257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:25.420257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:25.420257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:25.420257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:25.420257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:25.420257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:25.420257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:25.426006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 429/854] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:25.427402 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ab361-73eb-4934-8d8f-39d0781f4a16 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:25.507596 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81d468-9479-43a2-beeb-81edbc834924 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:25.508882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:25.524455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:25.547825 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edb5c2-75e6-4970-9587-2a4d6f4c3228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:25.548649 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edb5c2-75e6-4970-9587-2a4d6f4c3228 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:25.549113 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edb5c2-75e6-4970-9587-2a4d6f4c3228 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:25.549515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edb5c2-75e6-4970-9587-2a4d6f4c3228 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:25.550351 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edb5c2-75e6-4970-9587-2a4d6f4c3228 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:25.550762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edb5c2-75e6-4970-9587-2a4d6f4c3228 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:25.552252 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edb5c2-75e6-4970-9587-2a4d6f4c3228 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:25.554709 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ab361-73eb-4934-8d8f-39d0781f4a16 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:25.579828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 426/855] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:25.581036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edb5c2-75e6-4970-9587-2a4d6f4c3228 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:25.605182 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b8b39-c421-4ca0-82e1-f6a0214bc60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:25.606318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b8b39-c421-4ca0-82e1-f6a0214bc60d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:25.606848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b8b39-c421-4ca0-82e1-f6a0214bc60d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:25.607290 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b8b39-c421-4ca0-82e1-f6a0214bc60d None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:25.608329 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b8b39-c421-4ca0-82e1-f6a0214bc60d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:25.608831 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b8b39-c421-4ca0-82e1-f6a0214bc60d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:25.610764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b8b39-c421-4ca0-82e1-f6a0214bc60d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:25.626962 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 430/856] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:17:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:25.633955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b8b39-c421-4ca0-82e1-f6a0214bc60d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:25.709985 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edb5c2-75e6-4970-9587-2a4d6f4c3228 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:25.750728 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a43cf-026e-44d3-ab92-ae02e549b002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:25.751588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a43cf-026e-44d3-ab92-ae02e549b002 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:25.751947 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a43cf-026e-44d3-ab92-ae02e549b002 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:25.752301 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a43cf-026e-44d3-ab92-ae02e549b002 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:25.761721 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a43cf-026e-44d3-ab92-ae02e549b002 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:25.763529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a43cf-026e-44d3-ab92-ae02e549b002 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:25.775206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b8b39-c421-4ca0-82e1-f6a0214bc60d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:25.780453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-5b3a43cf-026e-44d3-ab92-ae02e549b002 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 06:17:25.780453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:25.780453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:25.780453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:25.780453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:25.780453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:25.780453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:25.780453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:25.780453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:25.780453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:25.780453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:25.780453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:25.780453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:17:25.780453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:17:25.780453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:17:25.780453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:25.781514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:25.782479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:25.782479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:25.782479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:17:25.782479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:17:25.782479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:17:25.782479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:17:25.782479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:17:25.782479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:17:25.782479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 06:17:25.782479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:25.785877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 427/857] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 06:17:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:25.788033 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a43cf-026e-44d3-ab92-ae02e549b002 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:25.817721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:25.829993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:25.831517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63daa9ed-5c11-4b3b-a0bb-e7d71d29a629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:25.832328 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daa9ed-5c11-4b3b-a0bb-e7d71d29a629 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:25.832520 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daa9ed-5c11-4b3b-a0bb-e7d71d29a629 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:25.833007 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daa9ed-5c11-4b3b-a0bb-e7d71d29a629 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:25.834368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63daa9ed-5c11-4b3b-a0bb-e7d71d29a629 None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=5fc205d053d54506bb6c9227e4a1b044) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:25.834742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63daa9ed-5c11-4b3b-a0bb-e7d71d29a629 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:25.836849 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63daa9ed-5c11-4b3b-a0bb-e7d71d29a629 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:25.857415 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 431/858] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:17:25 2022] GET /identity/v3/role_assignments?user.id=5fc205d053d54506bb6c9227e4a1b044&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:25.857415 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63715ddc-63dc-4fcc-b17c-2b3fe72b4194 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:25.857415 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63715ddc-63dc-4fcc-b17c-2b3fe72b4194 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:25.857415 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63715ddc-63dc-4fcc-b17c-2b3fe72b4194 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:25.857415 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 432/859] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:25.858364 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f9a7d-e575-41ce-8ca9-22ce86624cd6 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:25.858657 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f9a7d-e575-41ce-8ca9-22ce86624cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:25.858939 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f9a7d-e575-41ce-8ca9-22ce86624cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:25.878204 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-373f9a7d-e575-41ce-8ca9-22ce86624cd6 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:25.879439 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-373f9a7d-e575-41ce-8ca9-22ce86624cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:25.944057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a43cf-026e-44d3-ab92-ae02e549b002 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:25.974926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 433/860] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:26.001899 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f9a7d-e575-41ce-8ca9-22ce86624cd6 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:26.014902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a5d31-020f-4d26-b388-f324feeeb9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:26.016072 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a5d31-020f-4d26-b388-f324feeeb9bd None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:26.016586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a5d31-020f-4d26-b388-f324feeeb9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:26.017141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a5d31-020f-4d26-b388-f324feeeb9bd None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:26.019871 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a5d31-020f-4d26-b388-f324feeeb9bd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:26.020433 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a5d31-020f-4d26-b388-f324feeeb9bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:26.023607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a5d31-020f-4d26-b388-f324feeeb9bd None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:26.061031 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 428/861] 10.222.0.65 () {56 vars in 1149 bytes} [Thu May 19 06:17:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:26.066358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a5d31-020f-4d26-b388-f324feeeb9bd None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:26.141824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f9a7d-e575-41ce-8ca9-22ce86624cd6 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:26.175375 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:26.193207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:26.195901 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285ecd0-d829-4682-ae5a-14167e88cf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:26.196856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285ecd0-d829-4682-ae5a-14167e88cf96 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:26.197127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285ecd0-d829-4682-ae5a-14167e88cf96 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:26.197460 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285ecd0-d829-4682-ae5a-14167e88cf96 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:26.198166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0285ecd0-d829-4682-ae5a-14167e88cf96 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:26.199525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0285ecd0-d829-4682-ae5a-14167e88cf96 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:26.204354 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a5d31-020f-4d26-b388-f324feeeb9bd None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:26.213217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-0285ecd0-d829-4682-ae5a-14167e88cf96 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 06:17:26.213217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:26.213217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:26.213217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:26.213217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:26.213217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:26.213217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:26.213217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:26.213217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 06:17:26.213217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:26.213217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:26.213217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:26.213217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:26.213217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:26.213217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:26.213217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:26.214507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:26.214507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:26.214507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:26.214507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:26.214507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:26.214507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:26.214507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:26.214507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:26.214507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:26.214507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:26.214507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:26.214507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:26.214507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:26.214507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:26.214507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:26.214507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:26.215446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:26.215446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:26.215446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:26.215446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:26.215446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:26.215446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:26.215446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:26.215446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:26.215446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:26.215446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:26.215446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:26.215446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:26.215446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:26.215446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 06:17:26.215446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:26.215446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 434/862] 10.222.0.65 () {56 vars in 1132 bytes} [Thu May 19 06:17:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:26.222361 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285ecd0-d829-4682-ae5a-14167e88cf96 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:26.266705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190daee4-bf78-45c1-954a-1b40a3dbebfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:26.267844 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190daee4-bf78-45c1-954a-1b40a3dbebfa None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:26.268310 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190daee4-bf78-45c1-954a-1b40a3dbebfa None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:26.268805 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190daee4-bf78-45c1-954a-1b40a3dbebfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:26.362216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285ecd0-d829-4682-ae5a-14167e88cf96 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:26.375994 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190daee4-bf78-45c1-954a-1b40a3dbebfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:26.377753 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190daee4-bf78-45c1-954a-1b40a3dbebfa None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:26.436930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf77503-39a2-45a1-9ad8-40b37eedd277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:26.437573 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf77503-39a2-45a1-9ad8-40b37eedd277 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:26.437867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf77503-39a2-45a1-9ad8-40b37eedd277 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:26.438213 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf77503-39a2-45a1-9ad8-40b37eedd277 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:26.438951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf77503-39a2-45a1-9ad8-40b37eedd277 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:26.439316 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf77503-39a2-45a1-9ad8-40b37eedd277 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:26.440743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf77503-39a2-45a1-9ad8-40b37eedd277 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:26.446645 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 435/863] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 06:17:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:26.453020 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf77503-39a2-45a1-9ad8-40b37eedd277 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:26.497176 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:26.520442 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 429/864] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:17:26 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:17:26.525447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:26.525879 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190daee4-bf78-45c1-954a-1b40a3dbebfa None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:26.608695 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf77503-39a2-45a1-9ad8-40b37eedd277 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:26.664201 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec69c5-7e76-483d-9389-136aaf8dff6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:26.664942 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec69c5-7e76-483d-9389-136aaf8dff6e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:26.665350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec69c5-7e76-483d-9389-136aaf8dff6e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:26.665637 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec69c5-7e76-483d-9389-136aaf8dff6e None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:26.666537 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec69c5-7e76-483d-9389-136aaf8dff6e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:26.666805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec69c5-7e76-483d-9389-136aaf8dff6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:26.668512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec69c5-7e76-483d-9389-136aaf8dff6e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:26.678712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 436/865] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 06:17:26 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:26.714548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190daee4-bf78-45c1-954a-1b40a3dbebfa None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:26.757555 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c489-a41b-4fb4-a4ca-a064f0fc55c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:26.758800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c489-a41b-4fb4-a4ca-a064f0fc55c5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:26.759279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c489-a41b-4fb4-a4ca-a064f0fc55c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:26.759777 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c489-a41b-4fb4-a4ca-a064f0fc55c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:26.773896 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1c489-a41b-4fb4-a4ca-a064f0fc55c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:26.775725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1c489-a41b-4fb4-a4ca-a064f0fc55c5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:26.797460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-45c1c489-a41b-4fb4-a4ca-a064f0fc55c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:26.797460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:26.797460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:26.797460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:26.797460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:26.797460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:26.797460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:26.797460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:26.797460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:26.797460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:26.797460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:26.797460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:26.797460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:26.797460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:26.797460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:26.798635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:26.798635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:26.798635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:26.798635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:26.798635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:26.798635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:26.798635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:26.798635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:26.798635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:26.798635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:26.798635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:26.798635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:26.798635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:26.798635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:26.798635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:26.798635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:26.798635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:26.799807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:26.799807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:26.799807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:26.799807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:26.801946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 430/866] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:26.808235 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c489-a41b-4fb4-a4ca-a064f0fc55c5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:26.808739 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984dde9-964f-4f26-8ee4-366d1578d982 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:26.809191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984dde9-964f-4f26-8ee4-366d1578d982 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:26.809309 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984dde9-964f-4f26-8ee4-366d1578d982 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:26.811016 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 437/867] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:26.816728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786d706-d8a3-4a8d-a054-a1df57055522 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:26.817017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786d706-d8a3-4a8d-a054-a1df57055522 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:26.817327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786d706-d8a3-4a8d-a054-a1df57055522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:26.835828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c786d706-d8a3-4a8d-a054-a1df57055522 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:26.837051 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c786d706-d8a3-4a8d-a054-a1df57055522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:26.879912 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:26.946633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 438/868] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:26.957255 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c489-a41b-4fb4-a4ca-a064f0fc55c5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:26.970013 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786d706-d8a3-4a8d-a054-a1df57055522 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:27.003242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd433eb-c801-4130-a3e4-fca03935862c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:27.004033 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd433eb-c801-4130-a3e4-fca03935862c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:27.004388 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd433eb-c801-4130-a3e4-fca03935862c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:27.004738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd433eb-c801-4130-a3e4-fca03935862c None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:27.005661 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd433eb-c801-4130-a3e4-fca03935862c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:27.006063 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd433eb-c801-4130-a3e4-fca03935862c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:27.007538 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd433eb-c801-4130-a3e4-fca03935862c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:27.024418 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 431/869] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:27.032288 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd433eb-c801-4130-a3e4-fca03935862c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:27.051657 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:27.123758 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786d706-d8a3-4a8d-a054-a1df57055522 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:27.177980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd433eb-c801-4130-a3e4-fca03935862c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:27.189065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44b55b-605b-48be-81f3-82a1abb7782d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:27.190191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44b55b-605b-48be-81f3-82a1abb7782d None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:27.190707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44b55b-605b-48be-81f3-82a1abb7782d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:27.191195 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44b55b-605b-48be-81f3-82a1abb7782d None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:27.192198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44b55b-605b-48be-81f3-82a1abb7782d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:27.193109 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:27.195987 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44b55b-605b-48be-81f3-82a1abb7782d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:27.222159 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 439/870] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 06:17:26 2022] POST /identity/v3/services => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:17:27.266025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f87d1c-5cf0-4153-a6fb-dedcc8c131fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:27.266913 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f87d1c-5cf0-4153-a6fb-dedcc8c131fb None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:27.267203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f87d1c-5cf0-4153-a6fb-dedcc8c131fb None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:27.267543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f87d1c-5cf0-4153-a6fb-dedcc8c131fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:27.280232 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f87d1c-5cf0-4153-a6fb-dedcc8c131fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:27.282371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f87d1c-5cf0-4153-a6fb-dedcc8c131fb None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:27.302320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-c8f87d1c-5cf0-4153-a6fb-dedcc8c131fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:27.302320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:27.302320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:27.302320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:27.302320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:27.302320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:27.302320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:27.302320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:27.302320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:27.302320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:27.302320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:27.302320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:27.302320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:27.302320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:27.302320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:27.303318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:27.303318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:27.303318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:27.303318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:27.303318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:27.303318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:27.303318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:27.303318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:27.303318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:27.303318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:27.303318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:27.303318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:27.303318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:27.303318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:27.303318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:27.303318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:27.303318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:27.304210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:27.304210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:27.304210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:27.304210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:27.304210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:27.304210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:27.304210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:27.304210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:27.304210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 432/871] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:27.312105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44b55b-605b-48be-81f3-82a1abb7782d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:27.456342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44b55b-605b-48be-81f3-82a1abb7782d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:27.459436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:27.501244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc491b0-d7cb-4fd2-9d0e-4a968a72f06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:27.502334 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc491b0-d7cb-4fd2-9d0e-4a968a72f06c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:27.502830 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc491b0-d7cb-4fd2-9d0e-4a968a72f06c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:27.503365 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc491b0-d7cb-4fd2-9d0e-4a968a72f06c None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:27.504371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc491b0-d7cb-4fd2-9d0e-4a968a72f06c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:27.504891 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc491b0-d7cb-4fd2-9d0e-4a968a72f06c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:27.506599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc491b0-d7cb-4fd2-9d0e-4a968a72f06c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:27.523294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 440/872] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:17:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:27.531332 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f87d1c-5cf0-4153-a6fb-dedcc8c131fb None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:27.647653 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f87d1c-5cf0-4153-a6fb-dedcc8c131fb None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:27.686190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fabb38f-d1e2-4623-b86a-21d54ede45f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:27.686871 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabb38f-d1e2-4623-b86a-21d54ede45f2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:27.687159 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabb38f-d1e2-4623-b86a-21d54ede45f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:27.687850 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabb38f-d1e2-4623-b86a-21d54ede45f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:27.688693 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabb38f-d1e2-4623-b86a-21d54ede45f2 None admin] RBAC: Adding query filter params (role.id=5cc9ca1bc48b4d749d21d7111b14e187, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=aee77a3ed91445ab8b7f58e7f8bc348a) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:27.689011 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabb38f-d1e2-4623-b86a-21d54ede45f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:27.690514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabb38f-d1e2-4623-b86a-21d54ede45f2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:27.698000 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 433/873] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:17:27 2022] GET /identity/v3/role_assignments?user.id=aee77a3ed91445ab8b7f58e7f8bc348a&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=5cc9ca1bc48b4d749d21d7111b14e187 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:28.850676 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e99de-c780-4384-aa83-923dbfb006ce None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:28.851192 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e99de-c780-4384-aa83-923dbfb006ce None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:28.851637 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e99de-c780-4384-aa83-923dbfb006ce None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:28.854654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 441/874] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:28.865065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfccec6-9e8e-4b44-9992-7d1ee57c8711 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:28.865605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfccec6-9e8e-4b44-9992-7d1ee57c8711 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:28.866051 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfccec6-9e8e-4b44-9992-7d1ee57c8711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:28.894102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3dfccec6-9e8e-4b44-9992-7d1ee57c8711 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:28.895799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3dfccec6-9e8e-4b44-9992-7d1ee57c8711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:28.919720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:28.975479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 434/875] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:28 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:28.998108 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e99de-c780-4384-aa83-923dbfb006ce None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:29.083135 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d45bc0-4a71-48bc-acea-0dd18ee1ee3a None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:29.083488 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d45bc0-4a71-48bc-acea-0dd18ee1ee3a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:29.083675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d45bc0-4a71-48bc-acea-0dd18ee1ee3a None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:29.085695 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 435/876] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:17:29.091266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863da478-3659-41f1-9561-bec95d57f627 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:29.091556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863da478-3659-41f1-9561-bec95d57f627 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:29.091830 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863da478-3659-41f1-9561-bec95d57f627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:29.109652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-863da478-3659-41f1-9561-bec95d57f627 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:29.110762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-863da478-3659-41f1-9561-bec95d57f627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:29.125300 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e99de-c780-4384-aa83-923dbfb006ce None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:29.167413 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f8ed2-a890-40e5-aa59-650f9ff7873e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:29.168634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f8ed2-a890-40e5-aa59-650f9ff7873e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:29.169217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f8ed2-a890-40e5-aa59-650f9ff7873e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:29.169745 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f8ed2-a890-40e5-aa59-650f9ff7873e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:29.170617 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f8ed2-a890-40e5-aa59-650f9ff7873e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:29.172345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f8ed2-a890-40e5-aa59-650f9ff7873e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:29.186722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f95f8ed2-a890-40e5-aa59-650f9ff7873e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 06:17:29.186722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:29.186722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:29.186722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:29.186722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:29.186722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:29.186722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:29.186722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:29.186722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 06:17:29.186722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:29.186722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:29.186722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:29.186722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:29.186722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:29.186722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:29.186722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:29.186722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:29.187790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:29.187790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:29.187790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:29.187790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:29.187790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:29.187790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:29.187790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:29.187790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:29.187790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:29.187790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:29.187790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:29.187790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:29.187790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:29.187790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:29.187790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:29.187790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:29.188610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:29.188610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:29.188610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:29.188610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:29.188610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:29.188610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:29.188610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:29.188610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:29.188610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:29.188610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:29.188610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:29.188610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:29.188610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 06:17:29.188610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:29.189379 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 436/877] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:29 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:29.191556 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 442/878] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 06:17:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:29.201948 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863da478-3659-41f1-9561-bec95d57f627 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:29.207164 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f8ed2-a890-40e5-aa59-650f9ff7873e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:29.239630 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:29.295369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:29.312423 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863da478-3659-41f1-9561-bec95d57f627 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:29.336714 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f8ed2-a890-40e5-aa59-650f9ff7873e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:29.351545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc46ecb-9fb7-4fbd-853e-79e22826ac12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:29.352242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc46ecb-9fb7-4fbd-853e-79e22826ac12 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:29.352526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc46ecb-9fb7-4fbd-853e-79e22826ac12 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:29.352807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc46ecb-9fb7-4fbd-853e-79e22826ac12 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:29.353751 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc46ecb-9fb7-4fbd-853e-79e22826ac12 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:29.354062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc46ecb-9fb7-4fbd-853e-79e22826ac12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:29.355738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc46ecb-9fb7-4fbd-853e-79e22826ac12 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:29.361800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 437/879] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 06:17:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:29.367373 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc46ecb-9fb7-4fbd-853e-79e22826ac12 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:29.381172 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bdb947-995d-4551-81cb-643813794972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:29.382255 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdb947-995d-4551-81cb-643813794972 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:29.382774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdb947-995d-4551-81cb-643813794972 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:29.383288 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdb947-995d-4551-81cb-643813794972 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:29.384252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bdb947-995d-4551-81cb-643813794972 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:29.386551 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bdb947-995d-4551-81cb-643813794972 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:29.404930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 443/880] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 06:17:29 2022] POST /identity/v3/services => generated 269 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:17:29.504139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc46ecb-9fb7-4fbd-853e-79e22826ac12 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:29.555538 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ab454-a2b8-4544-800b-6dfadbb1d281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:29.556225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ab454-a2b8-4544-800b-6dfadbb1d281 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:29.557027 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ab454-a2b8-4544-800b-6dfadbb1d281 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:29.559711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ab454-a2b8-4544-800b-6dfadbb1d281 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:29.560482 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ab454-a2b8-4544-800b-6dfadbb1d281 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:29.560790 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ab454-a2b8-4544-800b-6dfadbb1d281 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:29.562238 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ab454-a2b8-4544-800b-6dfadbb1d281 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:29.570249 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 438/881] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 06:17:29 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:29.735151 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abdb65-60ef-41c5-8b6a-e9cb9d94d814 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:29.735527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abdb65-60ef-41c5-8b6a-e9cb9d94d814 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:29.735762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abdb65-60ef-41c5-8b6a-e9cb9d94d814 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:29.737618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 444/882] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:29.743592 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8efb1f-232b-4d82-9c9c-023b684a504b None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:29.743911 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8efb1f-232b-4d82-9c9c-023b684a504b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:29.744199 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8efb1f-232b-4d82-9c9c-023b684a504b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:29.763668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-8d8efb1f-232b-4d82-9c9c-023b684a504b None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:29.764978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8d8efb1f-232b-4d82-9c9c-023b684a504b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:29.767381 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:29.853932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 439/883] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:29.875354 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57abdb65-60ef-41c5-8b6a-e9cb9d94d814 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:29.991559 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:30.008807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57abdb65-60ef-41c5-8b6a-e9cb9d94d814 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:30.059594 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e39c9-f75f-422d-9ae5-7e59ad881a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:30.060905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e39c9-f75f-422d-9ae5-7e59ad881a52 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:30.061450 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e39c9-f75f-422d-9ae5-7e59ad881a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:30.061990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e39c9-f75f-422d-9ae5-7e59ad881a52 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:30.062874 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e39c9-f75f-422d-9ae5-7e59ad881a52 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:30.064476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e39c9-f75f-422d-9ae5-7e59ad881a52 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:30.078152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-636e39c9-f75f-422d-9ae5-7e59ad881a52 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 06:17:30.078152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:30.078152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:30.078152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:30.078152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:30.078152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:30.078152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:30.078152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:30.078152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 06:17:30.078152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:30.078152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:30.078152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:30.078152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:30.078152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:30.078152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:30.078152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:30.078152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:30.079224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:30.079224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:30.079224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:30.079224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:30.079224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:30.079224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:30.079224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:30.079224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:30.079224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:30.079224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:30.079224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:30.079224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:30.079224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:30.079224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:30.079224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:30.079224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:30.080076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:30.080076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:30.080076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:30.080076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:30.080076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:30.080076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:30.080076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:30.080076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:30.080076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:30.080076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:30.080076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:30.080076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:30.080076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 06:17:30.080076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:30.083084 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 445/884] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:17:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:30.090316 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8efb1f-232b-4d82-9c9c-023b684a504b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:30.124129 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bda89-411b-421c-bfe3-d444df847663 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:30.124794 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bda89-411b-421c-bfe3-d444df847663 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:30.125346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bda89-411b-421c-bfe3-d444df847663 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:30.127211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 446/885] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:30.132363 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da142436-f1d3-44c7-bf49-c060733f3ad2 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:30.132654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da142436-f1d3-44c7-bf49-c060733f3ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:30.132937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da142436-f1d3-44c7-bf49-c060733f3ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:30.153766 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-da142436-f1d3-44c7-bf49-c060733f3ad2 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:30.155832 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-da142436-f1d3-44c7-bf49-c060733f3ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:30.224143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8efb1f-232b-4d82-9c9c-023b684a504b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:30.240451 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 447/886] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:30.263970 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da142436-f1d3-44c7-bf49-c060733f3ad2 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:30.280141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93dedb3-8a8a-40ea-b66b-48c5c589d143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:30.281561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dedb3-8a8a-40ea-b66b-48c5c589d143 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:30.282059 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dedb3-8a8a-40ea-b66b-48c5c589d143 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:30.282536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dedb3-8a8a-40ea-b66b-48c5c589d143 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:30.283493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93dedb3-8a8a-40ea-b66b-48c5c589d143 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:30.284026 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93dedb3-8a8a-40ea-b66b-48c5c589d143 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:30.285649 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93dedb3-8a8a-40ea-b66b-48c5c589d143 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:30.293119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 440/887] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:17:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:30.298535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93dedb3-8a8a-40ea-b66b-48c5c589d143 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:30.301421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:30.381271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:30.411960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da142436-f1d3-44c7-bf49-c060733f3ad2 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:30.432999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93dedb3-8a8a-40ea-b66b-48c5c589d143 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:30.456027 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2f993-cdb7-49d6-8419-bde2ad65b489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:30.457040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2f993-cdb7-49d6-8419-bde2ad65b489 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:30.457482 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2f993-cdb7-49d6-8419-bde2ad65b489 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:30.457867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2f993-cdb7-49d6-8419-bde2ad65b489 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:30.471514 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2f993-cdb7-49d6-8419-bde2ad65b489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:17:30.476496 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbd9c9-6132-4c1c-9a5e-2899c17c7b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:30.477401 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbd9c9-6132-4c1c-9a5e-2899c17c7b0c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:30.477856 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbd9c9-6132-4c1c-9a5e-2899c17c7b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:30.478313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbd9c9-6132-4c1c-9a5e-2899c17c7b0c None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:30.479683 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbd9c9-6132-4c1c-9a5e-2899c17c7b0c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:30.480136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbd9c9-6132-4c1c-9a5e-2899c17c7b0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:30.481096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-fdc2f993-cdb7-49d6-8419-bde2ad65b489 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 06:17:30.481096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:30.481096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:30.481096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:30.481096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:30.481096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:30.481096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:30.481096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:30.481096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:30.481096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:30.481096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:30.481096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:30.481096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:17:30.481096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:17:30.481096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:17:30.481096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:17:30.482071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:17:30.482071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:17:30.482071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:17:30.482071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:17:30.482071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:30.482071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:30.482071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:30.482071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:30.482071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:30.482071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:30.482071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:30.482071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:30.482071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:30.482071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:30.482071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:30.482071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:30.482913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:30.482913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:30.482913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:30.482913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:30.482913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:17:30.482913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:17:30.482913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:17:30.482913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:17:30.482913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:17:30.482913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:17:30.482913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 06:17:30.482913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:30.484391 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbd9c9-6132-4c1c-9a5e-2899c17c7b0c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:30.488017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 448/888] 10.222.0.65 () {56 vars in 1120 bytes} [Thu May 19 06:17:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:30.489469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2f993-cdb7-49d6-8419-bde2ad65b489 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:30.491263 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 441/889] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:17:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:30.499046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbd9c9-6132-4c1c-9a5e-2899c17c7b0c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:30.575044 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:30.602261 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2f993-cdb7-49d6-8419-bde2ad65b489 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:30.615126 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbd9c9-6132-4c1c-9a5e-2899c17c7b0c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:30.641996 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87edfe-4386-4925-b423-36c927a94f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:30.642809 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87edfe-4386-4925-b423-36c927a94f4e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:30.643455 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87edfe-4386-4925-b423-36c927a94f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:30.643572 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87edfe-4386-4925-b423-36c927a94f4e None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:30.644431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87edfe-4386-4925-b423-36c927a94f4e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:30.644807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87edfe-4386-4925-b423-36c927a94f4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:30.646512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87edfe-4386-4925-b423-36c927a94f4e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:30.655815 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82d2fa-b2ed-4017-9f58-78c881e692ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:30.656810 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82d2fa-b2ed-4017-9f58-78c881e692ca None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:30.657305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82d2fa-b2ed-4017-9f58-78c881e692ca None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:30.657751 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82d2fa-b2ed-4017-9f58-78c881e692ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:30.658756 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82d2fa-b2ed-4017-9f58-78c881e692ca None admin] RBAC: Adding query filter params (interface=public, service_id=3ab0d820a5f64013a5f7d955b9b3d9a3, region_id=RegionOne) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:30.659276 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82d2fa-b2ed-4017-9f58-78c881e692ca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:30.660916 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82d2fa-b2ed-4017-9f58-78c881e692ca None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:30.661893 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 449/890] 10.222.0.65 () {56 vars in 1115 bytes} [Thu May 19 06:17:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:30.663446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87edfe-4386-4925-b423-36c927a94f4e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:30.668123 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:30.676446 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 442/891] 10.222.0.65 () {56 vars in 1251 bytes} [Thu May 19 06:17:30 2022] GET /identity/v3/endpoints?service_id=3ab0d820a5f64013a5f7d955b9b3d9a3&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:30.677787 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82d2fa-b2ed-4017-9f58-78c881e692ca None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:30.788612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87edfe-4386-4925-b423-36c927a94f4e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:30.801426 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82d2fa-b2ed-4017-9f58-78c881e692ca None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:30.831491 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33454d-136c-4a98-a532-f62545c7ccb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:30.832224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33454d-136c-4a98-a532-f62545c7ccb2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:30.832509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33454d-136c-4a98-a532-f62545c7ccb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:30.832785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33454d-136c-4a98-a532-f62545c7ccb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:30.837062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:30.844177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c781d5-2b30-4896-b24f-64a41cc82c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:30.845261 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c781d5-2b30-4896-b24f-64a41cc82c02 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:30.845822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c781d5-2b30-4896-b24f-64a41cc82c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:30.846307 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c781d5-2b30-4896-b24f-64a41cc82c02 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:30.847279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c781d5-2b30-4896-b24f-64a41cc82c02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:30.848999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c781d5-2b30-4896-b24f-64a41cc82c02 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:30.849322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33454d-136c-4a98-a532-f62545c7ccb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:30.851078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33454d-136c-4a98-a532-f62545c7ccb2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:30.857583 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 443/892] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 06:17:30 2022] GET /identity/v3/services => generated 1268 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:17:30.872358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-3f33454d-136c-4a98-a532-f62545c7ccb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:30.872358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:30.872358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:30.872358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:30.872358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:30.872358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:30.872358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:30.872358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:30.872358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:30.872358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:30.872358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:30.872358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:30.872358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:30.872358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:30.872358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:30.873370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:30.873370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:30.873370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:30.873370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:30.873370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:30.873370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:30.873370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:30.873370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:30.873370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:30.873370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:30.873370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:30.873370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:30.873370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:30.873370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:30.873370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:30.873370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:30.873370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:30.874271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:30.874271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:30.874271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:30.874271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:30.878683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 450/893] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:30.880508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c781d5-2b30-4896-b24f-64a41cc82c02 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:31.011079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c781d5-2b30-4896-b24f-64a41cc82c02 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:31.057834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1d68d-371c-4d0c-9bad-1c197b093d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:31.058956 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1d68d-371c-4d0c-9bad-1c197b093d8f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:31.059492 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1d68d-371c-4d0c-9bad-1c197b093d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:31.059980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1d68d-371c-4d0c-9bad-1c197b093d8f None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:31.060946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1d68d-371c-4d0c-9bad-1c197b093d8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:31.061512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1d68d-371c-4d0c-9bad-1c197b093d8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:31.063253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1d68d-371c-4d0c-9bad-1c197b093d8f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:31.082470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 444/894] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:31.084706 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33454d-136c-4a98-a532-f62545c7ccb2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:31.153339 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:31.208812 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33454d-136c-4a98-a532-f62545c7ccb2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:31.266559 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace7c4a-8f4b-472b-a18d-8d0d1a8b9c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:31.267975 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace7c4a-8f4b-472b-a18d-8d0d1a8b9c5a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:31.268558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace7c4a-8f4b-472b-a18d-8d0d1a8b9c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:31.269060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace7c4a-8f4b-472b-a18d-8d0d1a8b9c5a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:31.284087 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eace7c4a-8f4b-472b-a18d-8d0d1a8b9c5a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:31.285594 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eace7c4a-8f4b-472b-a18d-8d0d1a8b9c5a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:31.302094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-eace7c4a-8f4b-472b-a18d-8d0d1a8b9c5a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 06:17:31.302094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:31.302094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:31.302094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:31.302094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:31.302094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:31.302094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:31.302094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:31.302094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:31.302094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:31.302094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:31.302094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:31.302094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:17:31.302094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:17:31.302094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:17:31.302094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:31.303713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:31.305822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:31.305822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:31.305822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:17:31.305822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:17:31.305822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:17:31.305822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:17:31.305822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:17:31.305822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:17:31.305822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 06:17:31.305822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:31.305822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 451/895] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 06:17:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:31.310632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1d68d-371c-4d0c-9bad-1c197b093d8f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:31.424098 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:31.508141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1d68d-371c-4d0c-9bad-1c197b093d8f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:31.568887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c082183-5ede-452d-831b-d17e8d19607e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:31.569637 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c082183-5ede-452d-831b-d17e8d19607e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:31.569927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c082183-5ede-452d-831b-d17e8d19607e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:31.570200 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c082183-5ede-452d-831b-d17e8d19607e None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:31.571040 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c082183-5ede-452d-831b-d17e8d19607e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:31.571351 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c082183-5ede-452d-831b-d17e8d19607e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:31.573366 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c082183-5ede-452d-831b-d17e8d19607e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:31.593859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 445/896] 10.222.0.65 () {56 vars in 1149 bytes} [Thu May 19 06:17:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:31.603043 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace7c4a-8f4b-472b-a18d-8d0d1a8b9c5a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:31.791805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace7c4a-8f4b-472b-a18d-8d0d1a8b9c5a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:31.821471 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:31.854668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e9de7-b918-417b-bf4d-40f235fed5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:31.855556 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e9de7-b918-417b-bf4d-40f235fed5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:31.856017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e9de7-b918-417b-bf4d-40f235fed5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:31.856474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e9de7-b918-417b-bf4d-40f235fed5c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:31.992087 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e9de7-b918-417b-bf4d-40f235fed5c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:31.995744 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e9de7-b918-417b-bf4d-40f235fed5c6 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:32.093843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4951d-08cc-40e1-b0f1-fa01776147c9 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:32.094246 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4951d-08cc-40e1-b0f1-fa01776147c9 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:32.094388 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4951d-08cc-40e1-b0f1-fa01776147c9 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:32.096096 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 446/897] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:17:32.101580 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bb07f-0350-44cc-b5d1-85c6ab9e866b None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:32.102020 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bb07f-0350-44cc-b5d1-85c6ab9e866b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:32.102171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bb07f-0350-44cc-b5d1-85c6ab9e866b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:32.122622 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b28bb07f-0350-44cc-b5d1-85c6ab9e866b None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:32.124616 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b28bb07f-0350-44cc-b5d1-85c6ab9e866b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:32.189486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 452/898] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:17:31 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:17:32.194875 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e9de7-b918-417b-bf4d-40f235fed5c6 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:32.210402 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 447/899] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:32.222353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:32.237338 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bb07f-0350-44cc-b5d1-85c6ab9e866b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:32.284450 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:32.336462 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e9de7-b918-417b-bf4d-40f235fed5c6 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:32.355449 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bb07f-0350-44cc-b5d1-85c6ab9e866b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:32.383458 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccf492-d3c0-4531-9e93-76468ba14a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:32.384960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccf492-d3c0-4531-9e93-76468ba14a3b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:32.385253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccf492-d3c0-4531-9e93-76468ba14a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:32.385555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccf492-d3c0-4531-9e93-76468ba14a3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:32.394587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630992e-2d07-4c70-96e0-b001e29c7e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:32.395522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630992e-2d07-4c70-96e0-b001e29c7e7f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:32.395663 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630992e-2d07-4c70-96e0-b001e29c7e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:32.395939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630992e-2d07-4c70-96e0-b001e29c7e7f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:32.396599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630992e-2d07-4c70-96e0-b001e29c7e7f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:32.397897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdccf492-d3c0-4531-9e93-76468ba14a3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:32.398394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630992e-2d07-4c70-96e0-b001e29c7e7f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:32.399346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdccf492-d3c0-4531-9e93-76468ba14a3b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:32.411586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-7630992e-2d07-4c70-96e0-b001e29c7e7f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 06:17:32.411586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:32.411586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:32.411586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:32.411586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:32.411586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:32.411586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:32.411586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:32.411586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 06:17:32.411586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:32.411586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:32.411586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:32.411586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:32.411586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:32.411586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:32.411586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:32.412726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:32.412726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:32.412726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:32.412726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:32.412726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:32.412726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:32.412726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:32.412726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:32.412726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:32.412726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:32.412726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:32.412726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:32.412726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:32.412726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:32.412726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:32.412726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:32.414180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:32.414180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:32.414180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:32.414180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:32.414180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:32.414180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:32.414180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:32.414180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:32.414180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:32.414180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:32.414180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:32.414180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:32.414180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:32.414180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 06:17:32.414180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:32.414180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 448/900] 10.222.0.65 () {56 vars in 1132 bytes} [Thu May 19 06:17:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:32.415082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9698baf-c86f-44d3-9f8e-a2a918512829 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:32.415082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9698baf-c86f-44d3-9f8e-a2a918512829 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:32.415082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9698baf-c86f-44d3-9f8e-a2a918512829 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:32.416511 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 449/901] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:32.419180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9698baf-c86f-44d3-9f8e-a2a918512829 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:32.426486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-cdccf492-d3c0-4531-9e93-76468ba14a3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:32.426486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:32.426486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:32.426486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:32.426486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:32.426486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:32.426486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:32.426486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:32.426486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:32.426486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:32.426486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:32.426486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:32.426486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:32.426486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:32.426486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:32.427707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:32.427707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:32.427707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:32.427707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:32.427707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:32.427707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:32.427707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:32.427707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:32.427707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:32.427707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:32.427707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:32.427707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:32.427707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:32.427707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:32.427707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:32.427707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:32.427707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:32.428732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:32.428732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:32.428732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:32.428732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:32.428732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 453/902] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:32.429532 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab6417-e527-41fd-a710-fd9131949c8b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:32.429633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab6417-e527-41fd-a710-fd9131949c8b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:32.429930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab6417-e527-41fd-a710-fd9131949c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:32.448556 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-85ab6417-e527-41fd-a710-fd9131949c8b None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:32.449835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-85ab6417-e527-41fd-a710-fd9131949c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:32.533064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:32.540012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 454/903] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:32.541377 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab6417-e527-41fd-a710-fd9131949c8b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:32.543290 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9698baf-c86f-44d3-9f8e-a2a918512829 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:32.554512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:32.592823 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee79497-7ad2-42fa-8ed1-028479326567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:32.593524 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee79497-7ad2-42fa-8ed1-028479326567 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:32.594228 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee79497-7ad2-42fa-8ed1-028479326567 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:32.594228 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee79497-7ad2-42fa-8ed1-028479326567 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:32.594807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee79497-7ad2-42fa-8ed1-028479326567 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:32.595126 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee79497-7ad2-42fa-8ed1-028479326567 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:32.596523 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee79497-7ad2-42fa-8ed1-028479326567 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:32.602916 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 450/904] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 06:17:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:32.603664 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee79497-7ad2-42fa-8ed1-028479326567 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:32.668083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab6417-e527-41fd-a710-fd9131949c8b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:32.710064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f881d-5a51-4773-b6e9-a2c51bd6d860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:32.710770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f881d-5a51-4773-b6e9-a2c51bd6d860 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:32.711055 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f881d-5a51-4773-b6e9-a2c51bd6d860 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:32.711331 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f881d-5a51-4773-b6e9-a2c51bd6d860 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:32.712060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f881d-5a51-4773-b6e9-a2c51bd6d860 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:32.712365 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f881d-5a51-4773-b6e9-a2c51bd6d860 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:32.714351 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f881d-5a51-4773-b6e9-a2c51bd6d860 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:32.720532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee79497-7ad2-42fa-8ed1-028479326567 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:32.732226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 455/905] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:32.733251 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f881d-5a51-4773-b6e9-a2c51bd6d860 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:32.763491 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aebb91-87cf-4ed9-834f-b0418784a8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:32.764340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aebb91-87cf-4ed9-834f-b0418784a8b3 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:32.764636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aebb91-87cf-4ed9-834f-b0418784a8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:32.765251 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aebb91-87cf-4ed9-834f-b0418784a8b3 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:32.766315 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aebb91-87cf-4ed9-834f-b0418784a8b3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:32.768185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aebb91-87cf-4ed9-834f-b0418784a8b3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:32.787380 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 451/906] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 06:17:32 2022] POST /identity/v3/services => generated 241 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:17:32.788563 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aebb91-87cf-4ed9-834f-b0418784a8b3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:32.799253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:32.841142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:32.861796 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f881d-5a51-4773-b6e9-a2c51bd6d860 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:32.917573 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3dda9-dce7-4104-aa26-5312228e1882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:32.918404 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3dda9-dce7-4104-aa26-5312228e1882 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:32.918631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3dda9-dce7-4104-aa26-5312228e1882 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:32.918907 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3dda9-dce7-4104-aa26-5312228e1882 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:32.919641 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3dda9-dce7-4104-aa26-5312228e1882 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:32.919954 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3dda9-dce7-4104-aa26-5312228e1882 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:32.921469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3dda9-dce7-4104-aa26-5312228e1882 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:32.933890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 456/907] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 06:17:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:32.935357 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3dda9-dce7-4104-aa26-5312228e1882 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:32.960174 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aebb91-87cf-4ed9-834f-b0418784a8b3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:33.010552 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b39ab-22df-4762-aa94-79846a9e7c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:33.011543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b39ab-22df-4762-aa94-79846a9e7c35 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:33.011876 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b39ab-22df-4762-aa94-79846a9e7c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:33.012621 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b39ab-22df-4762-aa94-79846a9e7c35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:33.022097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b39ab-22df-4762-aa94-79846a9e7c35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:33.023107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b39ab-22df-4762-aa94-79846a9e7c35 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:33.042624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-0c2b39ab-22df-4762-aa94-79846a9e7c35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:33.042624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:33.042624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:33.042624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:33.042624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:33.042624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:33.042624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:33.042624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:33.042624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:33.042624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:33.042624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:33.042624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:33.042624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:33.042624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:33.042624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:33.043597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:33.043597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:33.043597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:33.043597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:33.043597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:33.043597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:33.043597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:33.043597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:33.043597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:33.043597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:33.043597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:33.043597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:33.043597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:33.043597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:33.043597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:33.043597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:33.043597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:33.044575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:33.044575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:33.044575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:33.044575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:33.044575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:33.044575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:33.044575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:33.044575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:33.045840 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 452/908] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:33.052363 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b39ab-22df-4762-aa94-79846a9e7c35 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:33.083274 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3dda9-dce7-4104-aa26-5312228e1882 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:33.105825 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:33.127072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd92c56-4926-4506-a55f-777dc14fc234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:33.128235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd92c56-4926-4506-a55f-777dc14fc234 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:33.128733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd92c56-4926-4506-a55f-777dc14fc234 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:33.129228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd92c56-4926-4506-a55f-777dc14fc234 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:33.130384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd92c56-4926-4506-a55f-777dc14fc234 None admin] RBAC: Adding query filter params (interface=public, service_id=7b20c48ae41d4c7995efd9a6405cc82a, region_id=RegionOne) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:33.131102 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd92c56-4926-4506-a55f-777dc14fc234 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:33.132720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd92c56-4926-4506-a55f-777dc14fc234 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:33.140171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 457/909] 10.222.0.65 () {56 vars in 1251 bytes} [Thu May 19 06:17:32 2022] GET /identity/v3/endpoints?service_id=7b20c48ae41d4c7995efd9a6405cc82a&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:33.146216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd92c56-4926-4506-a55f-777dc14fc234 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:33.189207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b39ab-22df-4762-aa94-79846a9e7c35 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:33.198235 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:33.233251 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb35d3-0338-47c1-8f9d-8e82826b8231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:33.234364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb35d3-0338-47c1-8f9d-8e82826b8231 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:33.235381 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb35d3-0338-47c1-8f9d-8e82826b8231 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:33.235701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb35d3-0338-47c1-8f9d-8e82826b8231 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:33.237530 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb35d3-0338-47c1-8f9d-8e82826b8231 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:33.238345 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb35d3-0338-47c1-8f9d-8e82826b8231 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:33.241848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb35d3-0338-47c1-8f9d-8e82826b8231 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:33.265268 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 453/910] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:17:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:33.266425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb35d3-0338-47c1-8f9d-8e82826b8231 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:33.267487 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd92c56-4926-4506-a55f-777dc14fc234 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:33.313086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf3048-9969-45ab-adf9-a79fcbbf3b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:33.314268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf3048-9969-45ab-adf9-a79fcbbf3b16 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:33.314821 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf3048-9969-45ab-adf9-a79fcbbf3b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:33.315485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf3048-9969-45ab-adf9-a79fcbbf3b16 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:33.316538 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf3048-9969-45ab-adf9-a79fcbbf3b16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:33.318679 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf3048-9969-45ab-adf9-a79fcbbf3b16 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:33.341743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 458/911] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 06:17:33 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:17:33.392298 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb35d3-0338-47c1-8f9d-8e82826b8231 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:33.459644 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c28f24-562d-4148-9542-c6a062b7a308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:33.460570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c28f24-562d-4148-9542-c6a062b7a308 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:33.461008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c28f24-562d-4148-9542-c6a062b7a308 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:33.461504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c28f24-562d-4148-9542-c6a062b7a308 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:33.463603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c28f24-562d-4148-9542-c6a062b7a308 None admin] RBAC: Adding query filter params (role.id=1a4655a3aaf244afb3988418b88d766f, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=aee77a3ed91445ab8b7f58e7f8bc348a) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:33.464098 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c28f24-562d-4148-9542-c6a062b7a308 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:33.466638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c28f24-562d-4148-9542-c6a062b7a308 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:33.477671 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 454/912] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:17:33 2022] GET /identity/v3/role_assignments?user.id=aee77a3ed91445ab8b7f58e7f8bc348a&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=1a4655a3aaf244afb3988418b88d766f => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:33.792191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe86817-fff9-4c44-8230-a438d4ae7b07 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:33.792503 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe86817-fff9-4c44-8230-a438d4ae7b07 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:33.792673 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe86817-fff9-4c44-8230-a438d4ae7b07 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:33.794443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 459/913] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:33.800411 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb681e45-0f74-444a-a968-415a307c0479 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:33.800735 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb681e45-0f74-444a-a968-415a307c0479 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:33.800991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb681e45-0f74-444a-a968-415a307c0479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:33.820506 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-eb681e45-0f74-444a-a968-415a307c0479 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:33.821774 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-eb681e45-0f74-444a-a968-415a307c0479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:33.833576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:33.909941 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 455/914] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:33.933464 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe86817-fff9-4c44-8230-a438d4ae7b07 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:33.994443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:34.065320 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe86817-fff9-4c44-8230-a438d4ae7b07 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:34.105219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc2d99-cefb-4762-933f-f3f54898636b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:34.106086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc2d99-cefb-4762-933f-f3f54898636b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:34.106374 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc2d99-cefb-4762-933f-f3f54898636b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:34.106656 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc2d99-cefb-4762-933f-f3f54898636b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:34.107329 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc2d99-cefb-4762-933f-f3f54898636b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:34.108815 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc2d99-cefb-4762-933f-f3f54898636b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:34.122113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-d0dc2d99-cefb-4762-933f-f3f54898636b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 06:17:34.122113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:34.122113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:34.122113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:34.122113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:34.122113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:34.122113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:34.122113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:34.122113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 06:17:34.122113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:34.122113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:34.122113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:34.122113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:34.122113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:34.122113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:34.122113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:34.122113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:34.123391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:34.123391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:34.123391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:34.123391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:34.123391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:34.123391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:34.123391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:34.123391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:34.123391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:34.123391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:34.123391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:34.123391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:34.123391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:34.123391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:34.123391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:34.123391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:34.124457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:34.124457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:34.124457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:34.124457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:34.124457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:34.124457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:34.124457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:34.124457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:34.124457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:34.124457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:34.124457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:34.124457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:34.124457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 06:17:34.124457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:34.124457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 460/915] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:17:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:34.129545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb681e45-0f74-444a-a968-415a307c0479 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:34.318486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb681e45-0f74-444a-a968-415a307c0479 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:34.402010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a431011-2628-438d-8599-977fe444bc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:34.403165 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a431011-2628-438d-8599-977fe444bc8c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:34.403665 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a431011-2628-438d-8599-977fe444bc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:34.404410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a431011-2628-438d-8599-977fe444bc8c None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:34.405779 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a431011-2628-438d-8599-977fe444bc8c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:34.406344 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a431011-2628-438d-8599-977fe444bc8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:34.408981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a431011-2628-438d-8599-977fe444bc8c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:34.419149 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 456/916] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:17:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:34.424457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc2d99-cefb-4762-933f-f3f54898636b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:34.562367 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc2d99-cefb-4762-933f-f3f54898636b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:34.578252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:34.602747 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d039e7-a67d-4a19-bb25-4e9818ad1d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:34.604076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d039e7-a67d-4a19-bb25-4e9818ad1d40 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:34.604365 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d039e7-a67d-4a19-bb25-4e9818ad1d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:34.604648 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d039e7-a67d-4a19-bb25-4e9818ad1d40 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:34.605406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d039e7-a67d-4a19-bb25-4e9818ad1d40 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:34.605716 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d039e7-a67d-4a19-bb25-4e9818ad1d40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:34.607104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d039e7-a67d-4a19-bb25-4e9818ad1d40 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:34.612951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 461/917] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 06:17:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:34.618622 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a431011-2628-438d-8599-977fe444bc8c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:34.633343 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:34.752614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a431011-2628-438d-8599-977fe444bc8c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:34.791970 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d17d16-bea1-4671-825b-3b2c947f0bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:34.792653 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d17d16-bea1-4671-825b-3b2c947f0bcc None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:34.792936 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d17d16-bea1-4671-825b-3b2c947f0bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:34.793214 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d17d16-bea1-4671-825b-3b2c947f0bcc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:34.794006 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d17d16-bea1-4671-825b-3b2c947f0bcc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:34.795399 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d17d16-bea1-4671-825b-3b2c947f0bcc None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:34.817539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 457/918] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 06:17:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:17:35.325017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5272acb-58ea-4b69-87f1-10433bbd6407 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:35.326080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5272acb-58ea-4b69-87f1-10433bbd6407 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:35.326580 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5272acb-58ea-4b69-87f1-10433bbd6407 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:35.328491 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 462/919] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:35.334061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d058e5-f281-4ed0-8367-121f052c8dea None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:35.334151 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d058e5-f281-4ed0-8367-121f052c8dea None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:35.334380 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d058e5-f281-4ed0-8367-121f052c8dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:35.352686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-94d058e5-f281-4ed0-8367-121f052c8dea None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:35.354357 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-94d058e5-f281-4ed0-8367-121f052c8dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:35.425747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:35.436586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 458/920] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:35.460352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5272acb-58ea-4b69-87f1-10433bbd6407 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:35.570282 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5272acb-58ea-4b69-87f1-10433bbd6407 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:35.606793 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcd209-a825-467e-8c11-427476146ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:35.607538 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcd209-a825-467e-8c11-427476146ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:35.607826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcd209-a825-467e-8c11-427476146ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:35.608105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcd209-a825-467e-8c11-427476146ff5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:35.608781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fcd209-a825-467e-8c11-427476146ff5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:35.610191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fcd209-a825-467e-8c11-427476146ff5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:35.624314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-b9fcd209-a825-467e-8c11-427476146ff5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 06:17:35.624314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:35.624314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:35.624314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:35.624314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:35.624314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:35.624314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:35.624314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:35.624314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 06:17:35.624314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:35.624314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:35.624314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:35.624314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:35.624314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:35.624314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:35.624314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:35.624314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:35.625577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:35.625577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:35.625577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:35.625577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:35.625577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:35.625577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:35.625577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:35.625577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:35.625577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:35.625577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:35.625577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:35.625577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:35.625577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:35.625577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:35.625577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:35.625577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:35.626900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:35.626900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:35.626900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:35.626900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:35.626900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:35.626900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:35.626900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:35.626900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:35.626900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:35.626900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:35.626900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:35.626900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:35.626900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 06:17:35.626900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:35.626900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 463/921] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 06:17:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:35.632330 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d058e5-f281-4ed0-8367-121f052c8dea None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:35.751433 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d058e5-f281-4ed0-8367-121f052c8dea None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:35.789467 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138660c-77c0-4a95-b5bb-4a18b8cf4a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:35.790563 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138660c-77c0-4a95-b5bb-4a18b8cf4a6b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:35.791632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138660c-77c0-4a95-b5bb-4a18b8cf4a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:35.792209 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138660c-77c0-4a95-b5bb-4a18b8cf4a6b None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:35.793143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138660c-77c0-4a95-b5bb-4a18b8cf4a6b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:35.793678 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138660c-77c0-4a95-b5bb-4a18b8cf4a6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:35.795287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138660c-77c0-4a95-b5bb-4a18b8cf4a6b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:35.801130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 459/922] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 06:17:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:35.813333 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcd209-a825-467e-8c11-427476146ff5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:35.840001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7332b1d-ae7a-4345-8af1-c897d6b8c68c None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:35.840638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7332b1d-ae7a-4345-8af1-c897d6b8c68c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:35.841125 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7332b1d-ae7a-4345-8af1-c897d6b8c68c None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:35.843035 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 460/923] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:35.848573 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f09165-0d0a-4462-8b44-9d53dafb5df3 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:35.849151 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f09165-0d0a-4462-8b44-9d53dafb5df3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:35.849687 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f09165-0d0a-4462-8b44-9d53dafb5df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:35.861731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:35.871445 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-12f09165-0d0a-4462-8b44-9d53dafb5df3 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:35.873029 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-12f09165-0d0a-4462-8b44-9d53dafb5df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:35.919930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcd209-a825-467e-8c11-427476146ff5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:35.939005 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:35.958779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1577c-1264-4dcf-95d9-b4c19854fd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:35.959462 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1577c-1264-4dcf-95d9-b4c19854fd98 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:35.965092 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1577c-1264-4dcf-95d9-b4c19854fd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:35.965259 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 461/924] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:35.965376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1577c-1264-4dcf-95d9-b4c19854fd98 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:35.965376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1577c-1264-4dcf-95d9-b4c19854fd98 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:35.965376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1577c-1264-4dcf-95d9-b4c19854fd98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:35.965575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b38c69-3f7f-4a91-bb2e-343925d1afa9 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:35.965575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b38c69-3f7f-4a91-bb2e-343925d1afa9 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:35.965575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b38c69-3f7f-4a91-bb2e-343925d1afa9 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:35.965778 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1577c-1264-4dcf-95d9-b4c19854fd98 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:35.965847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 462/925] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:35.969306 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1143fa-2df4-4de6-a0cc-e27cdc78d946 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:35.969633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1143fa-2df4-4de6-a0cc-e27cdc78d946 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:35.969889 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1143fa-2df4-4de6-a0cc-e27cdc78d946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:35.971785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 464/926] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 06:17:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:35.977248 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1577c-1264-4dcf-95d9-b4c19854fd98 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:35.996629 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4c1143fa-2df4-4de6-a0cc-e27cdc78d946 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:35.997930 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4c1143fa-2df4-4de6-a0cc-e27cdc78d946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:36.077308 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 463/927] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:36.077308 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1143fa-2df4-4de6-a0cc-e27cdc78d946 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:36.097498 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1577c-1264-4dcf-95d9-b4c19854fd98 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:36.099129 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:36.139872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de029a30-ac1f-4bf6-b70e-e12a75ecf437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:36.140542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de029a30-ac1f-4bf6-b70e-e12a75ecf437 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:36.140820 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de029a30-ac1f-4bf6-b70e-e12a75ecf437 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:36.141097 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de029a30-ac1f-4bf6-b70e-e12a75ecf437 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:36.141926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de029a30-ac1f-4bf6-b70e-e12a75ecf437 None admin] RBAC: Adding query filter params (interface=public, service_id=de58d2e0dfce40fbbf7cce3d622020f2, region_id=RegionOne) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:36.142336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de029a30-ac1f-4bf6-b70e-e12a75ecf437 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:36.143619 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de029a30-ac1f-4bf6-b70e-e12a75ecf437 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:36.154177 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 465/928] 10.222.0.65 () {56 vars in 1251 bytes} [Thu May 19 06:17:35 2022] GET /identity/v3/endpoints?service_id=de58d2e0dfce40fbbf7cce3d622020f2&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:36.154177 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de029a30-ac1f-4bf6-b70e-e12a75ecf437 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:36.199224 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1143fa-2df4-4de6-a0cc-e27cdc78d946 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:36.216410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:36.241952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726e14c-3f67-4991-89ac-a28bea82411c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:36.242764 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726e14c-3f67-4991-89ac-a28bea82411c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:36.243047 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726e14c-3f67-4991-89ac-a28bea82411c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:36.243322 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726e14c-3f67-4991-89ac-a28bea82411c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:36.255139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726e14c-3f67-4991-89ac-a28bea82411c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:17:36.258948 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de029a30-ac1f-4bf6-b70e-e12a75ecf437 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:36.263833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-2726e14c-3f67-4991-89ac-a28bea82411c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 06:17:36.263833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:36.263833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:36.263833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:36.263833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:36.263833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:36.263833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:36.263833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:36.263833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:36.263833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:36.263833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:36.263833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:36.263833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:17:36.263833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:17:36.263833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:17:36.263833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:17:36.264976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:17:36.264976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:17:36.264976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:17:36.264976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:17:36.264976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:36.264976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:36.264976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:36.264976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:36.264976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:36.264976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:36.264976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:36.264976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:36.264976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:36.264976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:36.264976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:36.264976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:36.265935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:36.265935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:36.265935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:36.265935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:36.265935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:17:36.265935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:17:36.265935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:17:36.265935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:17:36.265935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:17:36.265935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:17:36.265935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 06:17:36.265935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:36.265935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 464/929] 10.222.0.65 () {56 vars in 1120 bytes} [Thu May 19 06:17:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:36.266667 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726e14c-3f67-4991-89ac-a28bea82411c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:36.300484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1b8a9-de71-4697-9338-2b386542e9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:36.301324 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1b8a9-de71-4697-9338-2b386542e9df None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:36.301615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1b8a9-de71-4697-9338-2b386542e9df None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:36.301894 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1b8a9-de71-4697-9338-2b386542e9df None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:36.302555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1b8a9-de71-4697-9338-2b386542e9df None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:36.303930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1b8a9-de71-4697-9338-2b386542e9df None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:36.316759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-78f1b8a9-de71-4697-9338-2b386542e9df None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 06:17:36.316759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:36.316759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:36.316759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:36.316759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:36.316759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:36.316759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:36.316759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:36.316759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 06:17:36.316759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:36.316759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:36.316759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:36.316759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:36.316759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:36.316759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:36.316759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:36.317781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:36.317781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:36.317781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:36.317781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:36.317781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:36.317781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:36.317781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:36.317781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:36.317781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:36.317781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:36.317781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:36.317781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:36.317781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:36.317781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:36.317781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:36.317781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:36.318650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:36.318650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:36.318650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:36.318650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:36.318650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:36.318650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:36.318650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:36.318650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:36.318650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:36.318650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:36.318650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:36.318650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:36.318650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:36.318650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 06:17:36.318650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:36.318650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 466/930] 10.222.0.65 () {56 vars in 1132 bytes} [Thu May 19 06:17:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:36.319558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1b8a9-de71-4697-9338-2b386542e9df None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:36.356444 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:36.387364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726e14c-3f67-4991-89ac-a28bea82411c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:36.424672 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805198d-5edf-4d67-b36a-39931067eebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:36.425256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805198d-5edf-4d67-b36a-39931067eebe None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:36.425570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805198d-5edf-4d67-b36a-39931067eebe None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:36.425844 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805198d-5edf-4d67-b36a-39931067eebe None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:36.426606 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805198d-5edf-4d67-b36a-39931067eebe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:36.427985 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805198d-5edf-4d67-b36a-39931067eebe None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:36.433866 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1b8a9-de71-4697-9338-2b386542e9df None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:36.441482 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 465/931] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 06:17:36 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:17:36.441482 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805198d-5edf-4d67-b36a-39931067eebe None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:36.488089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35cd9e-6bb2-412d-92d2-cd23bc3e6549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:36.488781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35cd9e-6bb2-412d-92d2-cd23bc3e6549 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:36.489064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35cd9e-6bb2-412d-92d2-cd23bc3e6549 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:36.489371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35cd9e-6bb2-412d-92d2-cd23bc3e6549 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:36.490139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35cd9e-6bb2-412d-92d2-cd23bc3e6549 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:36.490444 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35cd9e-6bb2-412d-92d2-cd23bc3e6549 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:36.492152 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:36.493350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35cd9e-6bb2-412d-92d2-cd23bc3e6549 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:36.510415 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 467/932] 10.222.0.65 () {56 vars in 1115 bytes} [Thu May 19 06:17:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:36.515921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35cd9e-6bb2-412d-92d2-cd23bc3e6549 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:36.575947 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805198d-5edf-4d67-b36a-39931067eebe None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:36.626803 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ed5c7-6aef-497f-b9e8-5051a5209e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:36.628075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ed5c7-6aef-497f-b9e8-5051a5209e7e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:36.628584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ed5c7-6aef-497f-b9e8-5051a5209e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:36.629243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ed5c7-6aef-497f-b9e8-5051a5209e7e None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:36.630553 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ed5c7-6aef-497f-b9e8-5051a5209e7e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:36.631071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ed5c7-6aef-497f-b9e8-5051a5209e7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:36.632812 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ed5c7-6aef-497f-b9e8-5051a5209e7e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:36.639006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:36.645458 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 466/933] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 06:17:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:36.648375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ed5c7-6aef-497f-b9e8-5051a5209e7e None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:36.660813 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35cd9e-6bb2-412d-92d2-cd23bc3e6549 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:36.706171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6a552-9fa3-427a-bd6c-0dcaa1f8cc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:36.707957 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6a552-9fa3-427a-bd6c-0dcaa1f8cc56 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:36.708272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6a552-9fa3-427a-bd6c-0dcaa1f8cc56 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:36.708556 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6a552-9fa3-427a-bd6c-0dcaa1f8cc56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:36.727284 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6a552-9fa3-427a-bd6c-0dcaa1f8cc56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:36.730163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6a552-9fa3-427a-bd6c-0dcaa1f8cc56 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:36.755289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-38e6a552-9fa3-427a-bd6c-0dcaa1f8cc56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:36.755289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:36.755289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:36.755289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:36.755289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:36.755289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:36.755289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:36.755289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:36.755289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:36.755289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:36.755289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:36.755289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:36.755289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:36.755289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:36.755289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:36.756275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:36.756275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:36.756275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:36.756275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:36.756275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:36.756275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:36.756275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:36.756275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:36.756275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:36.756275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:36.756275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:36.756275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:36.756275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:36.756275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:36.756275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:36.756275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:36.756275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:36.757224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:36.757224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:36.757224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:36.757224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:36.759207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 468/934] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:36.764886 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6a552-9fa3-427a-bd6c-0dcaa1f8cc56 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:36.776260 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ed5c7-6aef-497f-b9e8-5051a5209e7e None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:36.778097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:36.817406 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12404eb-2dff-4c52-820d-bb24a19d6117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:36.818086 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12404eb-2dff-4c52-820d-bb24a19d6117 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:36.818431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12404eb-2dff-4c52-820d-bb24a19d6117 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:36.819349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12404eb-2dff-4c52-820d-bb24a19d6117 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:36.819459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12404eb-2dff-4c52-820d-bb24a19d6117 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:36.819763 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12404eb-2dff-4c52-820d-bb24a19d6117 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:36.821155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12404eb-2dff-4c52-820d-bb24a19d6117 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:36.827446 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 467/935] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 06:17:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:36.833102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12404eb-2dff-4c52-820d-bb24a19d6117 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:36.893227 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6a552-9fa3-427a-bd6c-0dcaa1f8cc56 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:36.933068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881922f8-1630-4da5-83de-15c110feca39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:36.933813 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881922f8-1630-4da5-83de-15c110feca39 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:36.934101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881922f8-1630-4da5-83de-15c110feca39 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:36.934384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881922f8-1630-4da5-83de-15c110feca39 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:36.935119 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881922f8-1630-4da5-83de-15c110feca39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:36.935430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881922f8-1630-4da5-83de-15c110feca39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:36.936941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881922f8-1630-4da5-83de-15c110feca39 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:36.955495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 469/936] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:36.958537 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881922f8-1630-4da5-83de-15c110feca39 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:36.964154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:36.981688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12404eb-2dff-4c52-820d-bb24a19d6117 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:37.023961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b0353-6721-4b17-9d60-5595bad635ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:37.024623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b0353-6721-4b17-9d60-5595bad635ca None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:37.024959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b0353-6721-4b17-9d60-5595bad635ca None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:37.025367 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b0353-6721-4b17-9d60-5595bad635ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:37.026920 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b0353-6721-4b17-9d60-5595bad635ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:37.029946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b0353-6721-4b17-9d60-5595bad635ca None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:37.057463 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 468/937] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 06:17:36 2022] POST /identity/v3/endpoints => generated 347 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:17:37.105049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881922f8-1630-4da5-83de-15c110feca39 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:37.151637 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2b670-3852-41e1-8483-4aed1ddc9da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:37.152909 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2b670-3852-41e1-8483-4aed1ddc9da7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:37.154918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6925f-5c0b-49ad-83e9-dd77825546f1 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:37.155429 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6925f-5c0b-49ad-83e9-dd77825546f1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:37.155903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6925f-5c0b-49ad-83e9-dd77825546f1 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:37.157785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2b670-3852-41e1-8483-4aed1ddc9da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:37.158058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2b670-3852-41e1-8483-4aed1ddc9da7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:37.158251 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 469/938] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:37.166809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e530e37-ce8a-4477-8c0b-454ba5a674e8 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:37.167195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e530e37-ce8a-4477-8c0b-454ba5a674e8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:37.167486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e530e37-ce8a-4477-8c0b-454ba5a674e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:37.169302 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2b670-3852-41e1-8483-4aed1ddc9da7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:37.172463 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2b670-3852-41e1-8483-4aed1ddc9da7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:37.186993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9e530e37-ce8a-4477-8c0b-454ba5a674e8 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:37.188376 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9e530e37-ce8a-4477-8c0b-454ba5a674e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:37.195337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-20c2b670-3852-41e1-8483-4aed1ddc9da7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 06:17:37.195337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:37.195337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:37.195337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:37.195337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:37.195337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:37.195337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:37.195337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:37.195337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:37.195337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:37.195337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:37.195337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:37.195337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:17:37.195337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:17:37.195337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:17:37.195337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:37.196433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:37.197786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:37.197786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:37.197786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:17:37.197786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:17:37.197786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:17:37.197786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:17:37.197786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:17:37.197786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:17:37.197786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 06:17:37.197786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:37.200957 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 470/939] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 06:17:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:37.208058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2b670-3852-41e1-8483-4aed1ddc9da7 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:37.211470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:37.288734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 470/940] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:37.306651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:37.315774 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e530e37-ce8a-4477-8c0b-454ba5a674e8 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:37.392585 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2b670-3852-41e1-8483-4aed1ddc9da7 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:37.441133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff82fdf-e3f5-4d39-9f51-fa1e9e18c380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:37.442416 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff82fdf-e3f5-4d39-9f51-fa1e9e18c380 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:37.443140 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff82fdf-e3f5-4d39-9f51-fa1e9e18c380 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:37.444046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff82fdf-e3f5-4d39-9f51-fa1e9e18c380 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:37.445853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff82fdf-e3f5-4d39-9f51-fa1e9e18c380 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:37.446642 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff82fdf-e3f5-4d39-9f51-fa1e9e18c380 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:37.449981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff82fdf-e3f5-4d39-9f51-fa1e9e18c380 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:37.473163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 471/941] 10.222.0.65 () {56 vars in 1149 bytes} [Thu May 19 06:17:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:37.473646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e530e37-ce8a-4477-8c0b-454ba5a674e8 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:37.483147 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff82fdf-e3f5-4d39-9f51-fa1e9e18c380 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:37.527287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d09b5-95c8-4cfc-b8b1-4b86e37af025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:37.529095 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d09b5-95c8-4cfc-b8b1-4b86e37af025 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:37.529899 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d09b5-95c8-4cfc-b8b1-4b86e37af025 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:37.530395 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d09b5-95c8-4cfc-b8b1-4b86e37af025 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:37.531303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d09b5-95c8-4cfc-b8b1-4b86e37af025 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:37.533320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d09b5-95c8-4cfc-b8b1-4b86e37af025 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:37.536023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: WARNING keystone.server.flask.common [None req-bb4d09b5-95c8-4cfc-b8b1-4b86e37af025 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 06:17:37.559196 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.sql.core [None req-bb4d09b5-95c8-4cfc-b8b1-4b86e37af025 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 06:17:37.559196 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 06:17:37.559196 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [parameters: {'id': '0e66dc537f974b849eb332e10fd8bf5a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 06:17:37.559196 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80281) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 06:17:37.584975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-bb4d09b5-95c8-4cfc-b8b1-4b86e37af025 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0e66dc537f974b849eb332e10fd8bf5a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0e66dc537f974b849eb332e10fd8bf5a. May 19 06:17:37.584975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:37.584975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 06:17:37.584975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 06:17:37.584975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 06:17:37.584975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 06:17:37.584975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 06:17:37.584975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application result = self._query(query) May 19 06:17:37.584975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 06:17:37.584975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application conn.query(q) May 19 06:17:37.584975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 06:17:37.584975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 06:17:37.584975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application result.read() May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 06:17:37.586032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 06:17:37.586991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 06:17:37.586991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application next(self.gen) May 19 06:17:37.586991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 06:17:37.586991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application yield resource May 19 06:17:37.586991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 06:17:37.586991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application next(self.gen) May 19 06:17:37.586991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 06:17:37.586991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self.session.rollback() May 19 06:17:37.586991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 06:17:37.586991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self.force_reraise() May 19 06:17:37.586991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 06:17:37.586991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise self.value May 19 06:17:37.586991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 06:17:37.586991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 06:17:37.586991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 06:17:37.586991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application session.commit() May 19 06:17:37.586991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 06:17:37.588203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 06:17:37.588203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 06:17:37.588203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._prepare_impl() May 19 06:17:37.588203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 06:17:37.588203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self.session.flush() May 19 06:17:37.588203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 06:17:37.588203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._flush(objects) May 19 06:17:37.588203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 06:17:37.588203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 06:17:37.588203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 06:17:37.588203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application compat.raise_( May 19 06:17:37.588203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 06:17:37.588203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception May 19 06:17:37.588203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 06:17:37.588203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application flush_context.execute() May 19 06:17:37.588203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 06:17:37.588203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rec.execute(self) May 19 06:17:37.589225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 06:17:37.589225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 06:17:37.589225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 06:17:37.589225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 06:17:37.589225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 06:17:37.589225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 06:17:37.589225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 06:17:37.589225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 06:17:37.589225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 06:17:37.589225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 06:17:37.589225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 06:17:37.589225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 06:17:37.589225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 06:17:37.589225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 06:17:37.589225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 06:17:37.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 06:17:37.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 06:17:37.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception May 19 06:17:37.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 06:17:37.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 06:17:37.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 06:17:37.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 06:17:37.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 06:17:37.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application result = self._query(query) May 19 06:17:37.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 06:17:37.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application conn.query(q) May 19 06:17:37.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 06:17:37.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 06:17:37.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 06:17:37.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application result.read() May 19 06:17:37.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 06:17:37.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 06:17:37.591012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 06:17:37.591012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 06:17:37.591012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 06:17:37.591012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 06:17:37.591012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 06:17:37.591012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 06:17:37.591012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 06:17:37.591012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 06:17:37.591012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [parameters: {'id': '0e66dc537f974b849eb332e10fd8bf5a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 06:17:37.591012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 06:17:37.591012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:37.591012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:37.591012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:37.591012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:37.591012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 06:17:37.591839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 06:17:37.591839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 06:17:37.591839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 06:17:37.591839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 06:17:37.591839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:37.591839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:37.591839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:37.591839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:37.591839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:37.591839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:37.591839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:37.591839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:37.591839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:37.591839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:37.591839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:37.591839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:37.591839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:37.595955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:37.595955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 06:17:37.595955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 06:17:37.595955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:37.595955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:37.595955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 06:17:37.595955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 06:17:37.595955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0e66dc537f974b849eb332e10fd8bf5a. May 19 06:17:37.595955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:37.595955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 471/942] 10.222.0.65 () {60 vars in 1140 bytes} [Thu May 19 06:17:37 2022] POST /identity/v3/projects => generated 262 bytes in 276 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:37.599149 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d09b5-95c8-4cfc-b8b1-4b86e37af025 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:37.622901 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:37.648773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff82fdf-e3f5-4d39-9f51-fa1e9e18c380 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:37.676576 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:37.706524 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12534532-8c5e-485f-bcaa-7c4547924a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:37.707565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12534532-8c5e-485f-bcaa-7c4547924a21 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:37.707921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12534532-8c5e-485f-bcaa-7c4547924a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:37.708417 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12534532-8c5e-485f-bcaa-7c4547924a21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:37.726120 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12534532-8c5e-485f-bcaa-7c4547924a21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:37.727521 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12534532-8c5e-485f-bcaa-7c4547924a21 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:37.758677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d09b5-95c8-4cfc-b8b1-4b86e37af025 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:37.762494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-12534532-8c5e-485f-bcaa-7c4547924a21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:37.762494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:37.762494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:37.762494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:37.762494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:37.762494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:37.762494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:37.762494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:37.762494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:37.762494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:37.762494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:37.762494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:37.762494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:37.762494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:37.762494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:37.763431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:37.763431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:37.763431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:37.763431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:37.763431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:37.763431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:37.763431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:37.763431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:37.763431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:37.763431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:37.763431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:37.763431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:37.763431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:37.763431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:37.763431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:37.763431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:37.763431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:37.764345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:37.764345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:37.764345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:37.764345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:37.769241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 472/943] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:37.770720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12534532-8c5e-485f-bcaa-7c4547924a21 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:37.808248 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69231eda-323f-4596-8844-6e4611907dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:37.809909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69231eda-323f-4596-8844-6e4611907dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:37.810699 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69231eda-323f-4596-8844-6e4611907dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:37.811187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69231eda-323f-4596-8844-6e4611907dc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:37.821024 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69231eda-323f-4596-8844-6e4611907dc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:37.823537 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69231eda-323f-4596-8844-6e4611907dc7 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:37.842852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-69231eda-323f-4596-8844-6e4611907dc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:37.842852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:37.842852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:37.842852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:37.842852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:37.842852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:37.842852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:37.842852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:37.842852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:37.842852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:37.842852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:37.842852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:37.842852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:37.842852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:37.842852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:37.843832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:37.843832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:37.843832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:37.843832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:37.843832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:37.843832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:37.843832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:37.843832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:37.843832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:37.843832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:37.843832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:37.843832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:37.843832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:37.843832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:37.843832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:37.843832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:37.843832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:37.844909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:37.844909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:37.844909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:37.844909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:37.844909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:37.844909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:37.844909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:37.844909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:37.844909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 472/944] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:37.850315 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69231eda-323f-4596-8844-6e4611907dc7 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:37.918854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12534532-8c5e-485f-bcaa-7c4547924a21 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:37.949703 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:37.972128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc31df5-4873-43ba-b0c3-d2377b981e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:37.972844 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc31df5-4873-43ba-b0c3-d2377b981e7a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:37.973129 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc31df5-4873-43ba-b0c3-d2377b981e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:37.973448 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc31df5-4873-43ba-b0c3-d2377b981e7a None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:37.974453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc31df5-4873-43ba-b0c3-d2377b981e7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:37.974771 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc31df5-4873-43ba-b0c3-d2377b981e7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:37.976442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc31df5-4873-43ba-b0c3-d2377b981e7a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:37.992651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 473/945] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:38.001034 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc31df5-4873-43ba-b0c3-d2377b981e7a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:38.002559 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69231eda-323f-4596-8844-6e4611907dc7 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:38.053849 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bf325-db9e-48c9-b737-590f904152b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:38.054535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bf325-db9e-48c9-b737-590f904152b3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:38.054819 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bf325-db9e-48c9-b737-590f904152b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:38.055097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bf325-db9e-48c9-b737-590f904152b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:38.056737 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:38.056871 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bf325-db9e-48c9-b737-590f904152b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:38.057333 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bf325-db9e-48c9-b737-590f904152b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:38.061138 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bf325-db9e-48c9-b737-590f904152b3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:38.090327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 473/946] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 06:17:37 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:38.172341 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc31df5-4873-43ba-b0c3-d2377b981e7a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:38.215304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e66b3-134b-4623-a434-632ecd10a438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:38.216433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e66b3-134b-4623-a434-632ecd10a438 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:38.216901 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e66b3-134b-4623-a434-632ecd10a438 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:38.217456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e66b3-134b-4623-a434-632ecd10a438 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:38.225937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e66b3-134b-4623-a434-632ecd10a438 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:38.227759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e66b3-134b-4623-a434-632ecd10a438 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:38.244932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-6d6e66b3-134b-4623-a434-632ecd10a438 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:38.244932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:38.244932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:38.244932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:38.244932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:38.244932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:38.244932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:38.244932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:38.244932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:38.244932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:38.244932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:38.244932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:38.244932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:38.244932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:38.244932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:38.246170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:38.246170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:38.246170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:38.246170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:38.246170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:38.246170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:38.246170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:38.246170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:38.246170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:38.246170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:38.246170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:38.246170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:38.246170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:38.246170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:38.246170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:38.246170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:38.246170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:38.247763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:38.247763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:38.247763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:38.247763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:38.247763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:38.247763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:38.247763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:38.247763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:38.250079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 474/947] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:38.257003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bf325-db9e-48c9-b737-590f904152b3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:38.380344 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bf325-db9e-48c9-b737-590f904152b3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:38.414723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:38.425087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d1de8-8828-4083-81ce-2b41fdcd408c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:38.426286 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d1de8-8828-4083-81ce-2b41fdcd408c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:38.426843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d1de8-8828-4083-81ce-2b41fdcd408c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:38.427497 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d1de8-8828-4083-81ce-2b41fdcd408c None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:38.429266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d1de8-8828-4083-81ce-2b41fdcd408c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:38.429834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d1de8-8828-4083-81ce-2b41fdcd408c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:38.432164 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d1de8-8828-4083-81ce-2b41fdcd408c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:38.446377 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 474/948] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:17:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:38.455770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e66b3-134b-4623-a434-632ecd10a438 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:38.555190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:38.566778 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e66b3-134b-4623-a434-632ecd10a438 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:38.603759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e00b5-558c-4e09-90a0-5479c49de100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:38.604661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e00b5-558c-4e09-90a0-5479c49de100 None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:38.604946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e00b5-558c-4e09-90a0-5479c49de100 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:38.605416 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e00b5-558c-4e09-90a0-5479c49de100 None admin] PATH_INFO: `/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/aee77a3ed91445ab8b7f58e7f8bc348a/roles/1a4655a3aaf244afb3988418b88d766f` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:38.607760 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e00b5-558c-4e09-90a0-5479c49de100 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3d399dc95424b848b736c0d0298d9a9, user_id=aee77a3ed91445ab8b7f58e7f8bc348a, role_id=1a4655a3aaf244afb3988418b88d766f)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:38.609306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e00b5-558c-4e09-90a0-5479c49de100 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:38.628616 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 475/949] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:17:38 2022] PUT /identity/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/aee77a3ed91445ab8b7f58e7f8bc348a/roles/1a4655a3aaf244afb3988418b88d766f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:17:38.976021 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e5189-1163-4ec7-b80e-78225b1e33e5 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:38.976541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e5189-1163-4ec7-b80e-78225b1e33e5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:38.976621 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e5189-1163-4ec7-b80e-78225b1e33e5 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:38.978979 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 475/950] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:38.984549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0763e3-2474-40cf-973f-c1094164c298 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:38.984847 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0763e3-2474-40cf-973f-c1094164c298 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:38.985130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0763e3-2474-40cf-973f-c1094164c298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:39.004054 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-bb0763e3-2474-40cf-973f-c1094164c298 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:39.005245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bb0763e3-2474-40cf-973f-c1094164c298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:39.084251 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 476/951] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:39.104513 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e5189-1163-4ec7-b80e-78225b1e33e5 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:39.212178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e5189-1163-4ec7-b80e-78225b1e33e5 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:39.249868 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81816cd5-edf0-44ab-9156-519da7f657e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:39.250592 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81816cd5-edf0-44ab-9156-519da7f657e5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:39.250873 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81816cd5-edf0-44ab-9156-519da7f657e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:39.251147 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81816cd5-edf0-44ab-9156-519da7f657e5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:39.251797 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81816cd5-edf0-44ab-9156-519da7f657e5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:39.253166 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81816cd5-edf0-44ab-9156-519da7f657e5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:39.265240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-81816cd5-edf0-44ab-9156-519da7f657e5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 06:17:39.265240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:39.265240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:39.265240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:39.265240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:39.265240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:39.265240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:39.265240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:39.265240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 06:17:39.265240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:39.265240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:39.265240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:39.265240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:39.265240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:39.265240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:39.265240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:39.265240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:39.266407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:39.266407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:39.266407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:39.266407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:39.266407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:39.266407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:39.266407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:39.266407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:39.266407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:39.266407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:39.266407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:39.266407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:39.266407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:39.266407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:39.266407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:39.266407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:39.267390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:39.267390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:39.267390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:39.267390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:39.267390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:39.267390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:39.267390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:39.267390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:39.267390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:39.267390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:39.267390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:39.267390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:39.267390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 06:17:39.267390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:39.267390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 476/952] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 06:17:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:39.274695 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0763e3-2474-40cf-973f-c1094164c298 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:39.376198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:39.398861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0763e3-2474-40cf-973f-c1094164c298 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:39.439461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686bc95-3f7b-490c-8f7b-b5b9edaf450c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:39.440144 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686bc95-3f7b-490c-8f7b-b5b9edaf450c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:39.440432 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686bc95-3f7b-490c-8f7b-b5b9edaf450c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:39.440713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686bc95-3f7b-490c-8f7b-b5b9edaf450c None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:39.441471 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686bc95-3f7b-490c-8f7b-b5b9edaf450c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:39.441851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686bc95-3f7b-490c-8f7b-b5b9edaf450c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:39.443336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686bc95-3f7b-490c-8f7b-b5b9edaf450c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:39.454743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 477/953] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 06:17:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:39.456260 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81816cd5-edf0-44ab-9156-519da7f657e5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:39.524068 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:39.575500 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81816cd5-edf0-44ab-9156-519da7f657e5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:39.610820 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913c521-ecc0-4237-8155-abdb1be0cde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:39.611488 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913c521-ecc0-4237-8155-abdb1be0cde6 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:39.611779 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913c521-ecc0-4237-8155-abdb1be0cde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:39.612063 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913c521-ecc0-4237-8155-abdb1be0cde6 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:39.612781 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913c521-ecc0-4237-8155-abdb1be0cde6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:39.613092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913c521-ecc0-4237-8155-abdb1be0cde6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:39.614529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913c521-ecc0-4237-8155-abdb1be0cde6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:39.620379 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 477/954] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 06:17:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:39.625987 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686bc95-3f7b-490c-8f7b-b5b9edaf450c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:39.659327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a7b5e-322b-4911-ac2a-7ddd86335331 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:39.659327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a7b5e-322b-4911-ac2a-7ddd86335331 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:39.659327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a7b5e-322b-4911-ac2a-7ddd86335331 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:39.659767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 478/955] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:17:39.666178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f5c44-e602-4c97-b86b-7af6f8547e2b None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:39.666178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f5c44-e602-4c97-b86b-7af6f8547e2b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:39.666178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f5c44-e602-4c97-b86b-7af6f8547e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:39.685660 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a83f5c44-e602-4c97-b86b-7af6f8547e2b None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:39.687646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a83f5c44-e602-4c97-b86b-7af6f8547e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:39.752403 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686bc95-3f7b-490c-8f7b-b5b9edaf450c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:39.773476 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 479/956] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:39.797361 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f5c44-e602-4c97-b86b-7af6f8547e2b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:39.800559 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b41e8-62bb-426d-8832-ff5d9a9df903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:39.801231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b41e8-62bb-426d-8832-ff5d9a9df903 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:39.801548 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b41e8-62bb-426d-8832-ff5d9a9df903 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:39.801833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b41e8-62bb-426d-8832-ff5d9a9df903 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:39.802598 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b41e8-62bb-426d-8832-ff5d9a9df903 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:39.803985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b41e8-62bb-426d-8832-ff5d9a9df903 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:39.821481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:39.834070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 478/957] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 06:17:39 2022] POST /identity/v3/endpoints => generated 349 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:17:39.868407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:39.937059 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f5c44-e602-4c97-b86b-7af6f8547e2b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:39.987095 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362bc8f-27f4-465c-98d3-183b029d7b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:39.988792 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362bc8f-27f4-465c-98d3-183b029d7b0d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:39.989327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362bc8f-27f4-465c-98d3-183b029d7b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:39.989878 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362bc8f-27f4-465c-98d3-183b029d7b0d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:39.990837 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362bc8f-27f4-465c-98d3-183b029d7b0d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:39.992521 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362bc8f-27f4-465c-98d3-183b029d7b0d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:40.006526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-3362bc8f-27f4-465c-98d3-183b029d7b0d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 06:17:40.006526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:40.006526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:40.006526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:40.006526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:40.006526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:40.006526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:40.006526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:40.006526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 06:17:40.006526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:40.006526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:40.006526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:40.006526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:40.006526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:40.006526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:40.006526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:40.006526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:40.007646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:40.007646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:40.007646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:40.007646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:40.007646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:40.007646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:40.007646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:40.007646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:40.007646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:40.007646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:40.007646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:40.007646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:40.007646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:40.007646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:40.007646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:40.007646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:40.008541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:40.008541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:40.008541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:40.008541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:40.008541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:40.008541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:40.008541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:40.008541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:40.008541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:40.008541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:40.008541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:40.008541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:40.008541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 06:17:40.008541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:40.013952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 480/958] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:40.021002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b41e8-62bb-426d-8832-ff5d9a9df903 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:40.143847 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b41e8-62bb-426d-8832-ff5d9a9df903 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:40.183449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a9e79-93b0-4368-bab1-dec82dca6c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:40.184602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a9e79-93b0-4368-bab1-dec82dca6c34 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:40.185118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a9e79-93b0-4368-bab1-dec82dca6c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:40.185682 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a9e79-93b0-4368-bab1-dec82dca6c34 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:40.186798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a9e79-93b0-4368-bab1-dec82dca6c34 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:40.187333 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a9e79-93b0-4368-bab1-dec82dca6c34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:40.189001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a9e79-93b0-4368-bab1-dec82dca6c34 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:40.195279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 479/959] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 06:17:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:40.201439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362bc8f-27f4-465c-98d3-183b029d7b0d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:40.310861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362bc8f-27f4-465c-98d3-183b029d7b0d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:40.318918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:40.351012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b50218-df1c-4593-a5be-f472b66c7ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:40.351711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b50218-df1c-4593-a5be-f472b66c7ece None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:40.352000 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b50218-df1c-4593-a5be-f472b66c7ece None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:40.352281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b50218-df1c-4593-a5be-f472b66c7ece None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:40.353072 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b50218-df1c-4593-a5be-f472b66c7ece None admin] RBAC: Adding query filter params (type=compute) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:40.353771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b50218-df1c-4593-a5be-f472b66c7ece None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:40.354988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b50218-df1c-4593-a5be-f472b66c7ece None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:40.360936 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 481/960] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 06:17:40 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:40.544787 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c4381-86bb-4f27-a2be-f410166f7b25 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:40.544994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c4381-86bb-4f27-a2be-f410166f7b25 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:40.545226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c4381-86bb-4f27-a2be-f410166f7b25 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:40.546893 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 480/961] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:40.553693 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ee377-9910-43c0-9735-3e7ea4d7f997 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:40.554467 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ee377-9910-43c0-9735-3e7ea4d7f997 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:40.554967 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ee377-9910-43c0-9735-3e7ea4d7f997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:40.574185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f35ee377-9910-43c0-9735-3e7ea4d7f997 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:40.575784 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f35ee377-9910-43c0-9735-3e7ea4d7f997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:40.654340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 482/962] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:40.680163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c4381-86bb-4f27-a2be-f410166f7b25 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:40.761614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:40.813701 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c4381-86bb-4f27-a2be-f410166f7b25 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:40.856216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76025f-42f7-4a69-a634-189eefdd2f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:40.856907 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76025f-42f7-4a69-a634-189eefdd2f73 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:40.857197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76025f-42f7-4a69-a634-189eefdd2f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:40.857511 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76025f-42f7-4a69-a634-189eefdd2f73 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:40.858245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76025f-42f7-4a69-a634-189eefdd2f73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:40.859702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76025f-42f7-4a69-a634-189eefdd2f73 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:40.861046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: WARNING keystone.server.flask.common [None req-6b76025f-42f7-4a69-a634-189eefdd2f73 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 06:17:40.876317 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.sql.core [None req-6b76025f-42f7-4a69-a634-189eefdd2f73 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 06:17:40.876317 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 06:17:40.876317 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [parameters: {'user_id': 'dae5617e02a74959b0a63bd21678722d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 06:17:40.876317 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80280) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 06:17:40.885806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-6b76025f-42f7-4a69-a634-189eefdd2f73 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 06:17:40.885806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:40.885806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 06:17:40.885806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 06:17:40.885806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 06:17:40.885806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 06:17:40.885806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 06:17:40.885806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application result = self._query(query) May 19 06:17:40.885806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 06:17:40.885806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application conn.query(q) May 19 06:17:40.885806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 06:17:40.885806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 06:17:40.885806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 06:17:40.885806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application result.read() May 19 06:17:40.885806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 06:17:40.886851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application next(self.gen) May 19 06:17:40.887848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 06:17:40.887848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application yield resource May 19 06:17:40.887848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 06:17:40.887848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application next(self.gen) May 19 06:17:40.887848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 06:17:40.887848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.session.rollback() May 19 06:17:40.887848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 06:17:40.887848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.force_reraise() May 19 06:17:40.887848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 06:17:40.887848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise self.value May 19 06:17:40.887848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 06:17:40.887848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 06:17:40.887848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 06:17:40.887848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application session.commit() May 19 06:17:40.887848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 06:17:40.887848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 06:17:40.887848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 06:17:40.888735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._prepare_impl() May 19 06:17:40.888735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 06:17:40.888735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.session.flush() May 19 06:17:40.888735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 06:17:40.888735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._flush(objects) May 19 06:17:40.888735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 06:17:40.888735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 06:17:40.888735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 06:17:40.888735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application compat.raise_( May 19 06:17:40.888735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 06:17:40.888735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception May 19 06:17:40.888735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 06:17:40.888735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application flush_context.execute() May 19 06:17:40.888735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 06:17:40.888735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rec.execute(self) May 19 06:17:40.888735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 06:17:40.888735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 06:17:40.889685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 06:17:40.889685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 06:17:40.889685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 06:17:40.889685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 06:17:40.889685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 06:17:40.889685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 06:17:40.889685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 06:17:40.889685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 06:17:40.889685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 06:17:40.889685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 06:17:40.889685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 06:17:40.889685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 06:17:40.889685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 06:17:40.889685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 06:17:40.889685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 06:17:40.889685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception May 19 06:17:40.890635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 06:17:40.890635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 06:17:40.890635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 06:17:40.890635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 06:17:40.890635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 06:17:40.890635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application result = self._query(query) May 19 06:17:40.890635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 06:17:40.890635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application conn.query(q) May 19 06:17:40.890635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 06:17:40.890635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 06:17:40.890635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 06:17:40.890635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application result.read() May 19 06:17:40.890635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 06:17:40.890635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 06:17:40.890635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 06:17:40.890635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 06:17:40.891475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 06:17:40.891475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 06:17:40.891475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 06:17:40.891475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 06:17:40.891475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 06:17:40.891475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 06:17:40.891475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [parameters: {'user_id': 'dae5617e02a74959b0a63bd21678722d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 06:17:40.891475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 06:17:40.891475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:40.891475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:40.891475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:40.891475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:40.891475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:40.891475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:40.891475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:40.892262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:40.892262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:40.892262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:40.892262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:40.892262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:40.892262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:40.892262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:40.892262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 06:17:40.892262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 06:17:40.892262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:40.892262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:40.892262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:17:40.892262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:40.892262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:17:40.892262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:40.892262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 06:17:40.892262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 06:17:40.893240 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 06:17:40.893240 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 06:17:40.893240 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 06:17:40.893240 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 06:17:40.893240 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 06:17:40.893240 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:40.893240 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 481/963] 10.222.0.65 () {60 vars in 1131 bytes} [Thu May 19 06:17:40 2022] POST /identity/v3/users => generated 160 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:40.904516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ee377-9910-43c0-9735-3e7ea4d7f997 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:41.013697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b79b4f-754b-4a16-ab06-14237b41865a None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:41.014247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b79b4f-754b-4a16-ab06-14237b41865a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:41.014247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b79b4f-754b-4a16-ab06-14237b41865a None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:41.019812 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 482/964] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:41.023962 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:41.025949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012f3b1-5cfa-4068-a23d-401993cabcaf None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:41.026484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012f3b1-5cfa-4068-a23d-401993cabcaf None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:41.027071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012f3b1-5cfa-4068-a23d-401993cabcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:41.028937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ee377-9910-43c0-9735-3e7ea4d7f997 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:41.046285 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3012f3b1-5cfa-4068-a23d-401993cabcaf None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:41.047969 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3012f3b1-5cfa-4068-a23d-401993cabcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:41.086279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cc1f7-6b9d-4ded-af01-6b325ef8fa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:41.088806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cc1f7-6b9d-4ded-af01-6b325ef8fa5a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:41.089323 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cc1f7-6b9d-4ded-af01-6b325ef8fa5a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:41.089791 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cc1f7-6b9d-4ded-af01-6b325ef8fa5a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:41.099233 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cc1f7-6b9d-4ded-af01-6b325ef8fa5a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:41.101538 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cc1f7-6b9d-4ded-af01-6b325ef8fa5a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:41.124548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-da4cc1f7-6b9d-4ded-af01-6b325ef8fa5a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 06:17:41.124548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:41.124548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:41.124548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:41.124548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:41.124548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:41.124548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:41.124548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:41.124548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:41.124548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:41.124548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:41.124548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:41.124548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:17:41.124548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:17:41.124548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:17:41.124548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:41.125640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:41.126614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:41.126614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:41.126614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:17:41.126614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:17:41.126614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:17:41.126614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:17:41.126614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:17:41.126614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:17:41.126614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 06:17:41.126614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:41.128084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 483/965] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:17:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:41.133142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cc1f7-6b9d-4ded-af01-6b325ef8fa5a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:41.149310 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 483/966] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:41.168606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012f3b1-5cfa-4068-a23d-401993cabcaf None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:41.244713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:41.266514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cc1f7-6b9d-4ded-af01-6b325ef8fa5a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:41.307439 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012f3b1-5cfa-4068-a23d-401993cabcaf None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:41.313402 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e512f-8c57-4fdb-abe1-98a0925827b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:41.314614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e512f-8c57-4fdb-abe1-98a0925827b3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:41.315104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e512f-8c57-4fdb-abe1-98a0925827b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:41.315582 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e512f-8c57-4fdb-abe1-98a0925827b3 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:41.317057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e512f-8c57-4fdb-abe1-98a0925827b3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:41.317620 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e512f-8c57-4fdb-abe1-98a0925827b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:41.323055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e512f-8c57-4fdb-abe1-98a0925827b3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:41.358906 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319487b-e309-464b-9ab0-c6936cad9935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:41.359754 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319487b-e309-464b-9ab0-c6936cad9935 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:41.360037 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319487b-e309-464b-9ab0-c6936cad9935 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:41.360391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319487b-e309-464b-9ab0-c6936cad9935 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:41.375189 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319487b-e309-464b-9ab0-c6936cad9935 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:17:41.379987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 484/967] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 06:17:41 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:41.390086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f319487b-e309-464b-9ab0-c6936cad9935 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 06:17:41.390086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:41.390086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:41.390086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:41.390086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:41.390086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:41.390086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:41.390086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:41.390086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:41.390086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:41.390086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:41.390086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:41.390086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:17:41.390086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:17:41.390086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:17:41.390086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:17:41.391212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:17:41.391212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:17:41.391212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:17:41.391212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:17:41.391212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:41.391212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:41.391212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:41.391212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:41.391212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:41.391212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:41.391212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:41.391212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:41.391212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:41.391212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:41.391212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:41.391212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:41.392287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:41.392287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:41.392287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:41.392287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:41.392287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:17:41.392287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:17:41.392287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:17:41.392287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:17:41.392287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:17:41.392287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:17:41.392287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 06:17:41.392287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:41.397443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 484/968] 10.222.0.65 () {56 vars in 1120 bytes} [Thu May 19 06:17:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:41.405644 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e512f-8c57-4fdb-abe1-98a0925827b3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:41.432533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:41.540826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e512f-8c57-4fdb-abe1-98a0925827b3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:41.581189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff70fa-dcbf-4df8-b4ba-e09a41113903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:41.581921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff70fa-dcbf-4df8-b4ba-e09a41113903 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:41.582227 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff70fa-dcbf-4df8-b4ba-e09a41113903 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:41.582484 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff70fa-dcbf-4df8-b4ba-e09a41113903 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:41.583257 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff70fa-dcbf-4df8-b4ba-e09a41113903 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:41.583569 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff70fa-dcbf-4df8-b4ba-e09a41113903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:41.585003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff70fa-dcbf-4df8-b4ba-e09a41113903 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:41.596372 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 485/969] 10.222.0.65 () {56 vars in 1115 bytes} [Thu May 19 06:17:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:41.605496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319487b-e309-464b-9ab0-c6936cad9935 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:41.747526 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319487b-e309-464b-9ab0-c6936cad9935 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:41.767894 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:41.788435 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4db2b-1d7f-41fe-812e-be4d79d362ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:41.789322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4db2b-1d7f-41fe-812e-be4d79d362ae None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:41.789622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4db2b-1d7f-41fe-812e-be4d79d362ae None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:41.789902 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4db2b-1d7f-41fe-812e-be4d79d362ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:41.802101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4db2b-1d7f-41fe-812e-be4d79d362ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:41.803708 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4db2b-1d7f-41fe-812e-be4d79d362ae None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:41.822912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-42c4db2b-1d7f-41fe-812e-be4d79d362ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:41.822912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:41.822912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:41.822912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:41.822912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:41.822912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:41.822912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:41.822912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:41.822912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:41.822912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:41.822912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:41.822912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:41.822912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:41.822912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:41.822912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:41.823873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:41.823873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:41.823873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:41.823873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:41.823873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:41.823873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:41.823873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:41.823873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:41.823873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:41.823873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:41.823873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:41.823873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:41.823873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:41.823873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:41.823873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:41.823873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:41.823873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:41.824779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:41.824779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:41.824779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:41.824779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:41.824779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 485/970] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:41.830012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff70fa-dcbf-4df8-b4ba-e09a41113903 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:41.914156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:41.936008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff70fa-dcbf-4df8-b4ba-e09a41113903 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:41.970191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236801da-022b-4be0-a7aa-15cf528cc450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:41.970862 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236801da-022b-4be0-a7aa-15cf528cc450 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:41.971148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236801da-022b-4be0-a7aa-15cf528cc450 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:41.971428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236801da-022b-4be0-a7aa-15cf528cc450 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:41.972153 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236801da-022b-4be0-a7aa-15cf528cc450 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:41.972460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236801da-022b-4be0-a7aa-15cf528cc450 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:41.973878 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236801da-022b-4be0-a7aa-15cf528cc450 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:41.990959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 486/971] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:41.992424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4db2b-1d7f-41fe-812e-be4d79d362ae None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:42.099007 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4db2b-1d7f-41fe-812e-be4d79d362ae None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:42.135040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654bbdc-c8c6-4753-ab01-3a9729a6ece1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:42.135983 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654bbdc-c8c6-4753-ab01-3a9729a6ece1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:42.136278 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654bbdc-c8c6-4753-ab01-3a9729a6ece1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:42.136547 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654bbdc-c8c6-4753-ab01-3a9729a6ece1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:42.145458 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654bbdc-c8c6-4753-ab01-3a9729a6ece1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:42.146969 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654bbdc-c8c6-4753-ab01-3a9729a6ece1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:42.164174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-2654bbdc-c8c6-4753-ab01-3a9729a6ece1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 06:17:42.164174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:42.164174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:42.164174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:42.164174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:42.164174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:42.164174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:42.164174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:42.164174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:42.164174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:42.164174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:42.164174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:42.164174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:17:42.164174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:17:42.164174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:17:42.164174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:42.165216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:42.166267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:42.166267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:42.166267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:17:42.166267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:17:42.166267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:17:42.166267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:17:42.166267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:17:42.166267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:17:42.166267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 06:17:42.166267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:42.166267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 486/972] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 06:17:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:42.173107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236801da-022b-4be0-a7aa-15cf528cc450 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:42.278232 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236801da-022b-4be0-a7aa-15cf528cc450 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:42.314350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2c9ac-707a-44a7-8272-76446bcf2917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:42.315340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2c9ac-707a-44a7-8272-76446bcf2917 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:42.315990 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2c9ac-707a-44a7-8272-76446bcf2917 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:42.316456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2c9ac-707a-44a7-8272-76446bcf2917 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:42.317564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2c9ac-707a-44a7-8272-76446bcf2917 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:42.318065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2c9ac-707a-44a7-8272-76446bcf2917 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:42.323409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2c9ac-707a-44a7-8272-76446bcf2917 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:42.334640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:42.348949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 487/973] 10.222.0.65 () {56 vars in 1149 bytes} [Thu May 19 06:17:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:42.350362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654bbdc-c8c6-4753-ab01-3a9729a6ece1 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:42.358894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7570770-a35c-489f-abbe-57a62c40c504 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:42.359193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7570770-a35c-489f-abbe-57a62c40c504 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:42.359436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7570770-a35c-489f-abbe-57a62c40c504 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:42.360976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 488/974] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:17:42.366222 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6c991-07eb-4554-88f8-5de77ef7bccc None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:42.366518 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6c991-07eb-4554-88f8-5de77ef7bccc None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:42.366795 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6c991-07eb-4554-88f8-5de77ef7bccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:42.382714 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:42.386236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-7ba6c991-07eb-4554-88f8-5de77ef7bccc None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:42.387541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7ba6c991-07eb-4554-88f8-5de77ef7bccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:42.462623 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654bbdc-c8c6-4753-ab01-3a9729a6ece1 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:42.471050 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 489/975] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:42.494045 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6c991-07eb-4554-88f8-5de77ef7bccc None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:42.502457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af5ce5-1122-459d-b554-6b9024d99f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:42.503319 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af5ce5-1122-459d-b554-6b9024d99f76 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:42.503723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af5ce5-1122-459d-b554-6b9024d99f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:42.504013 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af5ce5-1122-459d-b554-6b9024d99f76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:42.585738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af5ce5-1122-459d-b554-6b9024d99f76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:42.588281 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af5ce5-1122-459d-b554-6b9024d99f76 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:42.605208 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6c991-07eb-4554-88f8-5de77ef7bccc None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:42.623137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:42.635230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:42.643993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65630f53-b5af-4ed6-be91-d727ceca24aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:42.644820 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65630f53-b5af-4ed6-be91-d727ceca24aa None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:42.645108 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65630f53-b5af-4ed6-be91-d727ceca24aa None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:42.645521 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65630f53-b5af-4ed6-be91-d727ceca24aa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:42.646078 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65630f53-b5af-4ed6-be91-d727ceca24aa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:42.647470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65630f53-b5af-4ed6-be91-d727ceca24aa None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:42.660536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-65630f53-b5af-4ed6-be91-d727ceca24aa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 06:17:42.660536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:42.660536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:42.660536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:42.660536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:42.660536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:42.660536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:42.660536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:42.660536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 06:17:42.660536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:42.660536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:42.660536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:42.660536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:42.660536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:42.660536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:42.660536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:42.660536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:42.661627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:42.661627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:42.661627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:42.661627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:42.661627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:42.661627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:42.661627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:42.661627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:42.661627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:42.661627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:42.661627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:42.661627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:42.661627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:42.661627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:42.661627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:42.661627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:42.662504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:42.662504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:42.662504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:42.662504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:42.662504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:42.662504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:42.662504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:42.662504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:42.662504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:42.662504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:42.662504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:42.662504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:42.662504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 06:17:42.662504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:42.662504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 490/976] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 06:17:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:42.669629 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65630f53-b5af-4ed6-be91-d727ceca24aa None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:42.718149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 487/977] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:17:42 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:17:42.723634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af5ce5-1122-459d-b554-6b9024d99f76 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:42.788692 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65630f53-b5af-4ed6-be91-d727ceca24aa None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:42.823258 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1e347-1ad2-4d9d-a98a-44a542163172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:42.823937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1e347-1ad2-4d9d-a98a-44a542163172 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:42.824229 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1e347-1ad2-4d9d-a98a-44a542163172 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:42.824519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1e347-1ad2-4d9d-a98a-44a542163172 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:42.825232 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1e347-1ad2-4d9d-a98a-44a542163172 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:42.825650 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1e347-1ad2-4d9d-a98a-44a542163172 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:42.827046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1e347-1ad2-4d9d-a98a-44a542163172 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:42.829430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af5ce5-1122-459d-b554-6b9024d99f76 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:42.832067 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 491/978] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 06:17:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:42.833432 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795f850-8b61-4c89-bbbc-700c4691d658 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:42.833728 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795f850-8b61-4c89-bbbc-700c4691d658 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:42.834054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795f850-8b61-4c89-bbbc-700c4691d658 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:42.835313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 492/979] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:42.838264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795f850-8b61-4c89-bbbc-700c4691d658 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:42.861344 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:42.870125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbbabf-40d2-4c22-9128-4c208a49267c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:42.870929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbbabf-40d2-4c22-9128-4c208a49267c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:42.871216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbbabf-40d2-4c22-9128-4c208a49267c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:42.871496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbbabf-40d2-4c22-9128-4c208a49267c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:42.882258 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbbabf-40d2-4c22-9128-4c208a49267c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:42.883727 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbbabf-40d2-4c22-9128-4c208a49267c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:42.896352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:42.902582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-19dbbabf-40d2-4c22-9128-4c208a49267c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:42.902582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:42.902582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:42.902582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:42.902582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:42.902582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:42.902582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:42.902582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:42.902582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:42.902582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:42.902582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:42.902582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:42.902582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:42.902582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:42.902582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:42.903552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:42.903552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:42.903552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:42.903552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:42.903552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:42.903552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:42.903552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:42.903552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:42.903552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:42.903552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:42.903552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:42.903552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:42.903552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:42.903552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:42.903552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:42.903552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:42.903552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:42.904876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:42.904876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:42.904876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:42.904876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:42.904876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 488/980] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:42.905603 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499c46b-49f0-41dc-83de-2be2043f2251 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:42.905889 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499c46b-49f0-41dc-83de-2be2043f2251 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:42.906168 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499c46b-49f0-41dc-83de-2be2043f2251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:42.922537 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8499c46b-49f0-41dc-83de-2be2043f2251 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:42.923723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8499c46b-49f0-41dc-83de-2be2043f2251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:42.947468 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795f850-8b61-4c89-bbbc-700c4691d658 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:42.981230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c6d19-dddf-4166-aa44-57f695ff96b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:42.981930 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c6d19-dddf-4166-aa44-57f695ff96b4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:42.982216 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c6d19-dddf-4166-aa44-57f695ff96b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:42.982496 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c6d19-dddf-4166-aa44-57f695ff96b4 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:42.983200 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c6d19-dddf-4166-aa44-57f695ff96b4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:42.983509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c6d19-dddf-4166-aa44-57f695ff96b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:42.984920 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c6d19-dddf-4166-aa44-57f695ff96b4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:42.989597 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 489/981] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:42.990433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499c46b-49f0-41dc-83de-2be2043f2251 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:42.995349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 493/982] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 06:17:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:42.996114 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c6d19-dddf-4166-aa44-57f695ff96b4 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:43.090547 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499c46b-49f0-41dc-83de-2be2043f2251 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:43.101082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:43.103178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c6d19-dddf-4166-aa44-57f695ff96b4 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:43.123314 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:43.126434 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fec92-0e6d-4d79-9795-b7b08ff62c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:43.127118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fec92-0e6d-4d79-9795-b7b08ff62c55 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:43.127409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fec92-0e6d-4d79-9795-b7b08ff62c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:43.127691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fec92-0e6d-4d79-9795-b7b08ff62c55 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:43.128424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9fec92-0e6d-4d79-9795-b7b08ff62c55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:43.128738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9fec92-0e6d-4d79-9795-b7b08ff62c55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:43.130155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9fec92-0e6d-4d79-9795-b7b08ff62c55 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:43.138630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb3c0a-fbc9-4f10-9bad-df796b6f0307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:43.139294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb3c0a-fbc9-4f10-9bad-df796b6f0307 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:43.139583 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb3c0a-fbc9-4f10-9bad-df796b6f0307 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:43.139863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb3c0a-fbc9-4f10-9bad-df796b6f0307 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:43.140662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb3c0a-fbc9-4f10-9bad-df796b6f0307 None admin] RBAC: Adding query filter params (interface=admin, service_id=de58d2e0dfce40fbbf7cce3d622020f2, region_id=RegionOne) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:43.140976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb3c0a-fbc9-4f10-9bad-df796b6f0307 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:43.142195 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 490/983] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:43.142394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb3c0a-fbc9-4f10-9bad-df796b6f0307 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:43.142996 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fec92-0e6d-4d79-9795-b7b08ff62c55 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:43.150990 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 494/984] 10.222.0.65 () {56 vars in 1249 bytes} [Thu May 19 06:17:42 2022] GET /identity/v3/endpoints?service_id=de58d2e0dfce40fbbf7cce3d622020f2&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:43.151762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb3c0a-fbc9-4f10-9bad-df796b6f0307 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:43.245721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fec92-0e6d-4d79-9795-b7b08ff62c55 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:43.251910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb3c0a-fbc9-4f10-9bad-df796b6f0307 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:43.281978 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9e4b8-2933-4fbe-a4d9-aeae06031c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:43.282649 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9e4b8-2933-4fbe-a4d9-aeae06031c66 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:43.282931 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9e4b8-2933-4fbe-a4d9-aeae06031c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:43.283215 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9e4b8-2933-4fbe-a4d9-aeae06031c66 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:43.283932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9e4b8-2933-4fbe-a4d9-aeae06031c66 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:43.285352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9e4b8-2933-4fbe-a4d9-aeae06031c66 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:43.287925 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86872832-ea60-42d1-a2d2-c69a9d2f81ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:43.288722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86872832-ea60-42d1-a2d2-c69a9d2f81ba None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:43.289010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86872832-ea60-42d1-a2d2-c69a9d2f81ba None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:43.289302 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86872832-ea60-42d1-a2d2-c69a9d2f81ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:43.297016 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86872832-ea60-42d1-a2d2-c69a9d2f81ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:43.298539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86872832-ea60-42d1-a2d2-c69a9d2f81ba None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:43.299502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 491/985] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 06:17:43 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:17:43.300213 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9e4b8-2933-4fbe-a4d9-aeae06031c66 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:43.314105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-86872832-ea60-42d1-a2d2-c69a9d2f81ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:43.314105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:43.314105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:43.314105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:43.314105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:43.314105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:43.314105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:43.314105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:43.314105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:43.314105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:43.314105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:43.314105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:43.314105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:43.314105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:43.314105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:43.315218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:43.315218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:43.315218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:43.315218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:43.315218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:43.315218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:43.315218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:43.315218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:43.315218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:43.315218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:43.315218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:43.315218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:43.315218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:43.315218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:43.315218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:43.315218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:43.315218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:43.316304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:43.316304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:43.316304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:43.316304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:43.316304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:43.316304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:43.316304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:43.316304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:43.316304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 495/986] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:43.326595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86872832-ea60-42d1-a2d2-c69a9d2f81ba None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:43.347302 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:43.386176 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:43.444438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9e4b8-2933-4fbe-a4d9-aeae06031c66 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:43.454757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86872832-ea60-42d1-a2d2-c69a9d2f81ba None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:43.483320 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f631d4-baf8-478c-8d09-2c51dd8ea5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:43.484005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f631d4-baf8-478c-8d09-2c51dd8ea5bd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:43.484289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f631d4-baf8-478c-8d09-2c51dd8ea5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:43.484572 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f631d4-baf8-478c-8d09-2c51dd8ea5bd None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:43.485315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f631d4-baf8-478c-8d09-2c51dd8ea5bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:43.486697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f631d4-baf8-478c-8d09-2c51dd8ea5bd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:43.494317 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 492/987] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 06:17:43 2022] GET /identity/v3/services => generated 1727 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:17:43.501008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda5b47-b625-4087-98cf-9094801b2fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:43.501753 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda5b47-b625-4087-98cf-9094801b2fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:43.502049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda5b47-b625-4087-98cf-9094801b2fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:43.502334 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda5b47-b625-4087-98cf-9094801b2fa4 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:43.503177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda5b47-b625-4087-98cf-9094801b2fa4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:43.504465 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda5b47-b625-4087-98cf-9094801b2fa4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:43.506714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda5b47-b625-4087-98cf-9094801b2fa4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:43.522349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 496/988] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:17:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:43.528870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f631d4-baf8-478c-8d09-2c51dd8ea5bd None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:43.633230 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:43.646156 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f631d4-baf8-478c-8d09-2c51dd8ea5bd None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:43.683854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18709f-6d09-49a7-9df3-048a5ccacac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:43.684523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18709f-6d09-49a7-9df3-048a5ccacac9 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:43.684809 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18709f-6d09-49a7-9df3-048a5ccacac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:43.685087 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18709f-6d09-49a7-9df3-048a5ccacac9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:43.685967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18709f-6d09-49a7-9df3-048a5ccacac9 None admin] RBAC: Adding query filter params (role.id=1a4655a3aaf244afb3988418b88d766f, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=aee77a3ed91445ab8b7f58e7f8bc348a) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:43.686281 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18709f-6d09-49a7-9df3-048a5ccacac9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:43.688028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18709f-6d09-49a7-9df3-048a5ccacac9 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:43.695655 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 493/989] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:17:43 2022] GET /identity/v3/role_assignments?user.id=aee77a3ed91445ab8b7f58e7f8bc348a&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=1a4655a3aaf244afb3988418b88d766f => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:45.635350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c010f-f640-4020-b92e-77ce2b0c6dcf None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:45.635859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c010f-f640-4020-b92e-77ce2b0c6dcf None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:45.635937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c010f-f640-4020-b92e-77ce2b0c6dcf None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:45.637459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 497/990] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:45.642636 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d81b68-5841-4978-9b88-6a297d175f44 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:45.643033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d81b68-5841-4978-9b88-6a297d175f44 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:45.643211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d81b68-5841-4978-9b88-6a297d175f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:45.660748 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-57d81b68-5841-4978-9b88-6a297d175f44 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:45.661964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-57d81b68-5841-4978-9b88-6a297d175f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:45.737655 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 494/991] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:45 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:45.760769 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c010f-f640-4020-b92e-77ce2b0c6dcf None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:45.766363 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01f793-adec-46d7-8435-5ccb1b757031 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:45.766885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01f793-adec-46d7-8435-5ccb1b757031 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:45.767376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01f793-adec-46d7-8435-5ccb1b757031 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:45.769199 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 495/992] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:45.774010 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d97ff-e137-4bcb-ba4f-e93c9c591833 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:45.775054 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d97ff-e137-4bcb-ba4f-e93c9c591833 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:45.775528 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d97ff-e137-4bcb-ba4f-e93c9c591833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:45.793212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-4a4d97ff-e137-4bcb-ba4f-e93c9c591833 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:45.794733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4a4d97ff-e137-4bcb-ba4f-e93c9c591833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:45.875008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c010f-f640-4020-b92e-77ce2b0c6dcf None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:45.877894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:45.882923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 496/993] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:45 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:45.907001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d97ff-e137-4bcb-ba4f-e93c9c591833 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:45.912082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f8508-629a-48bc-9df3-13c74890407c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:45.912882 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f8508-629a-48bc-9df3-13c74890407c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:45.913165 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f8508-629a-48bc-9df3-13c74890407c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:45.913473 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f8508-629a-48bc-9df3-13c74890407c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:45.914128 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f8508-629a-48bc-9df3-13c74890407c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:45.915553 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f8508-629a-48bc-9df3-13c74890407c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:45.922654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:45.927554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-914f8508-629a-48bc-9df3-13c74890407c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 06:17:45.927554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:45.927554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:45.927554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:45.927554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:45.927554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:45.927554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:45.927554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:45.927554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 06:17:45.927554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:45.927554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:45.927554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:45.927554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:45.927554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:45.927554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:45.927554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:45.927554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:45.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:45.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:45.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:45.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:45.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:45.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:45.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:45.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:45.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:45.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:45.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:45.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:45.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:45.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:45.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:45.928587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:45.934989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:45.934989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:45.934989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:45.934989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:45.934989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:45.934989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:45.934989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:45.934989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:45.934989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:45.934989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:45.934989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:45.934989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:45.934989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 06:17:45.934989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:45.934989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 498/994] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:45.938575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f8508-629a-48bc-9df3-13c74890407c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:46.024771 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d97ff-e137-4bcb-ba4f-e93c9c591833 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:46.044883 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f8508-629a-48bc-9df3-13c74890407c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:46.060317 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5366c6f-8155-46cd-8107-8fb87321e1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:46.061125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5366c6f-8155-46cd-8107-8fb87321e1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:46.061444 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5366c6f-8155-46cd-8107-8fb87321e1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:46.061720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5366c6f-8155-46cd-8107-8fb87321e1c3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:46.062387 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5366c6f-8155-46cd-8107-8fb87321e1c3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:46.063756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5366c6f-8155-46cd-8107-8fb87321e1c3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:46.076430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f5366c6f-8155-46cd-8107-8fb87321e1c3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 06:17:46.076430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:46.076430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:46.076430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:46.076430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:46.076430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:46.076430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:46.076430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:46.076430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 06:17:46.076430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:46.076430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:46.076430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:46.076430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:46.076430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:46.076430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:46.076430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:46.076430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:46.077558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:46.077558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:46.077558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:46.077558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:46.077558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:46.077558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:46.077558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:46.077558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:46.077558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:46.077558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:46.077558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:46.077558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:46.077558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:46.077558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:46.077558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:46.077558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:46.078423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:46.078423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:46.078423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:46.078423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:46.078423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:46.078423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:46.078423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:46.078423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:46.078423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:46.078423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:46.078423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:46.078423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:46.078423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 06:17:46.078423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:46.078423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 497/995] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 06:17:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:46.079238 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee6023-c288-46ff-afb2-7c81c67ede34 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:46.079238 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee6023-c288-46ff-afb2-7c81c67ede34 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:46.079462 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee6023-c288-46ff-afb2-7c81c67ede34 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:46.080731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 498/996] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:46.083467 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee6023-c288-46ff-afb2-7c81c67ede34 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:46.084369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557275a-5548-4ab6-b75e-c9fac2914a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:46.085076 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557275a-5548-4ab6-b75e-c9fac2914a2d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:46.085388 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557275a-5548-4ab6-b75e-c9fac2914a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:46.085675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557275a-5548-4ab6-b75e-c9fac2914a2d None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:46.086391 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557275a-5548-4ab6-b75e-c9fac2914a2d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:46.086701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557275a-5548-4ab6-b75e-c9fac2914a2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:46.088102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557275a-5548-4ab6-b75e-c9fac2914a2d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:46.093337 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 499/997] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 06:17:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:46.094391 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1727e-b829-4033-873a-df59dd17a2c3 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:46.094683 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1727e-b829-4033-873a-df59dd17a2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:46.094966 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1727e-b829-4033-873a-df59dd17a2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:46.111250 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-00f1727e-b829-4033-873a-df59dd17a2c3 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:46.112451 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-00f1727e-b829-4033-873a-df59dd17a2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:46.139923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:46.170329 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:46.184166 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 500/998] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:46.185221 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1727e-b829-4033-873a-df59dd17a2c3 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:46.202266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee6023-c288-46ff-afb2-7c81c67ede34 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:46.238107 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2928caa-a92a-4b76-92a1-96c9acc37cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:46.238779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2928caa-a92a-4b76-92a1-96c9acc37cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:46.239138 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2928caa-a92a-4b76-92a1-96c9acc37cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:46.239349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2928caa-a92a-4b76-92a1-96c9acc37cd3 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:46.240061 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2928caa-a92a-4b76-92a1-96c9acc37cd3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:46.240370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2928caa-a92a-4b76-92a1-96c9acc37cd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:46.241798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2928caa-a92a-4b76-92a1-96c9acc37cd3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:46.247743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 499/999] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 06:17:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:46.248532 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2928caa-a92a-4b76-92a1-96c9acc37cd3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:46.285010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1727e-b829-4033-873a-df59dd17a2c3 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:46.322448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea850bc4-5c1c-484d-880c-74353edb36dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:46.322448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea850bc4-5c1c-484d-880c-74353edb36dd None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:46.322448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea850bc4-5c1c-484d-880c-74353edb36dd None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:46.322448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea850bc4-5c1c-484d-880c-74353edb36dd None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:46.322898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea850bc4-5c1c-484d-880c-74353edb36dd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:46.322898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea850bc4-5c1c-484d-880c-74353edb36dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:46.323676 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea850bc4-5c1c-484d-880c-74353edb36dd None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:46.328982 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 501/1000] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 06:17:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:46.329620 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea850bc4-5c1c-484d-880c-74353edb36dd None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:46.357384 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:46.363671 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2928caa-a92a-4b76-92a1-96c9acc37cd3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:46.398219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e8aa2-4f12-48c9-835e-e1457dd85bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:46.399011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e8aa2-4f12-48c9-835e-e1457dd85bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:46.399295 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e8aa2-4f12-48c9-835e-e1457dd85bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:46.400177 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e8aa2-4f12-48c9-835e-e1457dd85bf2 None admin] PATH_INFO: `/v3/services/image` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:46.400844 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e8aa2-4f12-48c9-835e-e1457dd85bf2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:46.402255 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e8aa2-4f12-48c9-835e-e1457dd85bf2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:46.414136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-cc9e8aa2-4f12-48c9-835e-e1457dd85bf2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 06:17:46.414136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:46.414136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:46.414136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:46.414136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:46.414136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:46.414136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:46.414136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:46.414136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 06:17:46.414136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:46.414136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:46.414136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:46.414136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:46.414136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:46.414136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:46.414136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:46.414136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:46.415322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:46.415322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:46.415322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:46.415322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:46.415322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:46.415322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:46.415322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:46.415322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:46.415322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:46.415322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:46.415322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:46.415322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:46.415322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:46.415322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:46.415322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:46.415322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:46.416194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:46.416194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:46.416194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:46.416194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:46.416194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:46.416194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:46.416194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:46.416194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:46.416194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:46.416194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:46.416194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:46.416194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:46.416194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 06:17:46.416194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:46.416194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 500/1001] 10.222.0.65 () {56 vars in 1105 bytes} [Thu May 19 06:17:46 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:46.417614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e8aa2-4f12-48c9-835e-e1457dd85bf2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:46.427153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:46.432412 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea850bc4-5c1c-484d-880c-74353edb36dd None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:46.467223 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1a38d-7a3a-4301-b0de-037b0a331dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:46.467888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1a38d-7a3a-4301-b0de-037b0a331dfb None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:46.468200 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1a38d-7a3a-4301-b0de-037b0a331dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:46.468478 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1a38d-7a3a-4301-b0de-037b0a331dfb None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:46.469182 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1a38d-7a3a-4301-b0de-037b0a331dfb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:46.469516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1a38d-7a3a-4301-b0de-037b0a331dfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:46.470895 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1a38d-7a3a-4301-b0de-037b0a331dfb None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:46.475971 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 502/1002] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 06:17:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:46.476748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1a38d-7a3a-4301-b0de-037b0a331dfb None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:46.525699 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e8aa2-4f12-48c9-835e-e1457dd85bf2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:46.560423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57ccf1-c063-4bd1-9744-855973c635ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:46.561090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57ccf1-c063-4bd1-9744-855973c635ca None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:46.561406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57ccf1-c063-4bd1-9744-855973c635ca None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:46.561688 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57ccf1-c063-4bd1-9744-855973c635ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:46.562488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57ccf1-c063-4bd1-9744-855973c635ca None admin] RBAC: Adding query filter params (interface=public, service_id=f282066e332d4cbe87b5e4c27bf78b39, region_id=RegionOne) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:46.562806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57ccf1-c063-4bd1-9744-855973c635ca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:46.564183 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57ccf1-c063-4bd1-9744-855973c635ca None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:46.568309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:46.569492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 501/1003] 10.222.0.65 () {56 vars in 1251 bytes} [Thu May 19 06:17:46 2022] GET /identity/v3/endpoints?service_id=f282066e332d4cbe87b5e4c27bf78b39&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:46.570188 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57ccf1-c063-4bd1-9744-855973c635ca None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:46.583410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1a38d-7a3a-4301-b0de-037b0a331dfb None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:46.617263 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2d6dc-945a-495a-9557-46930282e433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:46.617953 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2d6dc-945a-495a-9557-46930282e433 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:46.618309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2d6dc-945a-495a-9557-46930282e433 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:46.618711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2d6dc-945a-495a-9557-46930282e433 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:46.619423 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2d6dc-945a-495a-9557-46930282e433 None admin] RBAC: Adding query filter params (name=image) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:46.619732 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2d6dc-945a-495a-9557-46930282e433 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:46.621102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2d6dc-945a-495a-9557-46930282e433 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:46.625984 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 503/1004] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:46.626771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2d6dc-945a-495a-9557-46930282e433 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:46.640320 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:46.672485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57ccf1-c063-4bd1-9744-855973c635ca None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:46.706817 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99e9f5-30d4-4652-988a-d0535671829b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:46.707401 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99e9f5-30d4-4652-988a-d0535671829b None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:46.707687 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99e9f5-30d4-4652-988a-d0535671829b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:46.707967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99e9f5-30d4-4652-988a-d0535671829b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:46.708723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99e9f5-30d4-4652-988a-d0535671829b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:46.710150 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99e9f5-30d4-4652-988a-d0535671829b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:46.728399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 502/1005] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 06:17:46 2022] POST /identity/v3/endpoints => generated 325 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:17:46.729424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99e9f5-30d4-4652-988a-d0535671829b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:46.745636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2d6dc-945a-495a-9557-46930282e433 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:46.793456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df6d940-c4c2-4ae2-b85c-36eace2ace92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:46.794150 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6d940-c4c2-4ae2-b85c-36eace2ace92 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:46.794434 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6d940-c4c2-4ae2-b85c-36eace2ace92 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:46.794711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6d940-c4c2-4ae2-b85c-36eace2ace92 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:46.795419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df6d940-c4c2-4ae2-b85c-36eace2ace92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:46.796853 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df6d940-c4c2-4ae2-b85c-36eace2ace92 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:46.804166 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 504/1006] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 06:17:46 2022] GET /identity/v3/services => generated 1727 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:17:46.863450 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99e9f5-30d4-4652-988a-d0535671829b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:46.905176 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:46.914280 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e8752-e71d-40bb-81d5-574a602b739a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:46.915574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e8752-e71d-40bb-81d5-574a602b739a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:46.916324 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e8752-e71d-40bb-81d5-574a602b739a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:46.916606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e8752-e71d-40bb-81d5-574a602b739a None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:46.917412 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e8752-e71d-40bb-81d5-574a602b739a None admin] RBAC: Adding query filter params (type=image) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:46.917728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e8752-e71d-40bb-81d5-574a602b739a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:46.919213 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e8752-e71d-40bb-81d5-574a602b739a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:46.933031 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 503/1007] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:46 2022] GET /identity/v3/services?type=image => generated 122 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:49.069380 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f8431-4743-4d4f-a8c7-f7043c2b9be8 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:49.069961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f8431-4743-4d4f-a8c7-f7043c2b9be8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:49.069961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f8431-4743-4d4f-a8c7-f7043c2b9be8 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:49.071747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 505/1008] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:49.076940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da0d11-b37f-41e4-a5a6-8f93de950f69 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:49.077479 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da0d11-b37f-41e4-a5a6-8f93de950f69 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:49.077587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da0d11-b37f-41e4-a5a6-8f93de950f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:49.092132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf508e6-97cc-4c3b-ace9-1cd7a6a6b5c0 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:49.092419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf508e6-97cc-4c3b-ace9-1cd7a6a6b5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:49.092705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf508e6-97cc-4c3b-ace9-1cd7a6a6b5c0 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:49.094254 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 506/1009] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:49.095353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a8da0d11-b37f-41e4-a5a6-8f93de950f69 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:49.096546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a8da0d11-b37f-41e4-a5a6-8f93de950f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:49.098930 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46dbef-11f6-41fa-9e75-89d9e757b242 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:49.099222 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46dbef-11f6-41fa-9e75-89d9e757b242 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:49.099502 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46dbef-11f6-41fa-9e75-89d9e757b242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:49.116731 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-6b46dbef-11f6-41fa-9e75-89d9e757b242 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:49.117933 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6b46dbef-11f6-41fa-9e75-89d9e757b242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:49.127893 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:49.174620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 504/1010] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:49 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:49.191447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 507/1011] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:49 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:49.197620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da0d11-b37f-41e4-a5a6-8f93de950f69 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:49.215470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46dbef-11f6-41fa-9e75-89d9e757b242 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:49.304834 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da0d11-b37f-41e4-a5a6-8f93de950f69 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:49.318766 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46dbef-11f6-41fa-9e75-89d9e757b242 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:49.335087 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:49.346677 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c07238-8696-4215-8759-d4daf74ccb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:49.347470 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c07238-8696-4215-8759-d4daf74ccb14 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:49.347754 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c07238-8696-4215-8759-d4daf74ccb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:49.348035 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c07238-8696-4215-8759-d4daf74ccb14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:49.349197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c07238-8696-4215-8759-d4daf74ccb14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:49.350667 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c07238-8696-4215-8759-d4daf74ccb14 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:49.352946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 505/1012] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:49.354277 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343a017-cfe0-4027-b31e-e6d0894f1d13 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:49.354562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343a017-cfe0-4027-b31e-e6d0894f1d13 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:49.354835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343a017-cfe0-4027-b31e-e6d0894f1d13 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:49.356076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 506/1013] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:49.358413 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343a017-cfe0-4027-b31e-e6d0894f1d13 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:49.361039 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c85c3-e8fe-4461-9545-9b540ce480ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:49.361889 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c85c3-e8fe-4461-9545-9b540ce480ef None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:49.362177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c85c3-e8fe-4461-9545-9b540ce480ef None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:49.362456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c85c3-e8fe-4461-9545-9b540ce480ef None admin] PATH_INFO: `/v3/services/compute` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:49.363144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c85c3-e8fe-4461-9545-9b540ce480ef None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:49.364531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c85c3-e8fe-4461-9545-9b540ce480ef None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:49.377335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-b91c85c3-e8fe-4461-9545-9b540ce480ef None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 06:17:49.377335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:49.377335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:49.377335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:49.377335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:49.377335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:49.377335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:49.377335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:49.377335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 06:17:49.377335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:49.377335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:49.377335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:49.377335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:49.377335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:49.377335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:49.377335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:49.377335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:49.378591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:49.378591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:49.378591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:49.378591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:49.378591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:49.378591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:49.378591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:49.378591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:49.378591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:49.378591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:49.378591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:49.378591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:49.378591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:49.378591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:49.378591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:49.378591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:49.379460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:49.379460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:49.379460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:49.379460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:49.379460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:49.379460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:49.379460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:49.379460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:49.379460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:49.379460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:49.379460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:49.379460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:49.379460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 06:17:49.379460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:49.379460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 508/1014] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:49.380264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e16b0-9a48-49ab-98dd-b04f83838ea6 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:49.380264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e16b0-9a48-49ab-98dd-b04f83838ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:49.380264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e16b0-9a48-49ab-98dd-b04f83838ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:49.396908 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e17e16b0-9a48-49ab-98dd-b04f83838ea6 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:49.398138 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e17e16b0-9a48-49ab-98dd-b04f83838ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:49.400179 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:49.462120 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343a017-cfe0-4027-b31e-e6d0894f1d13 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:49.473891 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 509/1015] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:49 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:49.474738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e16b0-9a48-49ab-98dd-b04f83838ea6 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:49.502986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552dedc-4bb0-435a-bd92-50a2f2c75209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:49.503647 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552dedc-4bb0-435a-bd92-50a2f2c75209 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:49.503927 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552dedc-4bb0-435a-bd92-50a2f2c75209 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:49.504201 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552dedc-4bb0-435a-bd92-50a2f2c75209 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:49.504909 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552dedc-4bb0-435a-bd92-50a2f2c75209 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:49.506378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552dedc-4bb0-435a-bd92-50a2f2c75209 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:49.537193 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 507/1016] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 06:17:49 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:17:49.538028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552dedc-4bb0-435a-bd92-50a2f2c75209 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:49.592954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e16b0-9a48-49ab-98dd-b04f83838ea6 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:49.594957 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:49.635265 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed151a5-3e19-46e8-9875-2ca1498d132d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:49.636061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed151a5-3e19-46e8-9875-2ca1498d132d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:49.636414 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed151a5-3e19-46e8-9875-2ca1498d132d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:49.636738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed151a5-3e19-46e8-9875-2ca1498d132d None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:49.637477 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed151a5-3e19-46e8-9875-2ca1498d132d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:49.637772 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed151a5-3e19-46e8-9875-2ca1498d132d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:49.639201 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed151a5-3e19-46e8-9875-2ca1498d132d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:49.650795 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 510/1017] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 06:17:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:49.652171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed151a5-3e19-46e8-9875-2ca1498d132d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:49.655636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:49.660668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552dedc-4bb0-435a-bd92-50a2f2c75209 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:49.699181 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aaed92-050b-4bbd-a01e-4880d363bc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:49.699876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aaed92-050b-4bbd-a01e-4880d363bc1f None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:49.700163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aaed92-050b-4bbd-a01e-4880d363bc1f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:49.700444 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aaed92-050b-4bbd-a01e-4880d363bc1f None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:49.701200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aaed92-050b-4bbd-a01e-4880d363bc1f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:49.702718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aaed92-050b-4bbd-a01e-4880d363bc1f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:49.724907 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 508/1018] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 06:17:49 2022] POST /identity/v3/services => generated 242 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:17:49.791842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed151a5-3e19-46e8-9875-2ca1498d132d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:49.844813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08892ba6-2321-4a83-aa66-eec020295fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:49.845477 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892ba6-2321-4a83-aa66-eec020295fef None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:49.845770 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892ba6-2321-4a83-aa66-eec020295fef None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:49.846051 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892ba6-2321-4a83-aa66-eec020295fef None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:49.846769 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08892ba6-2321-4a83-aa66-eec020295fef None admin] RBAC: Adding query filter params (type=compute) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:49.847083 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08892ba6-2321-4a83-aa66-eec020295fef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:49.848499 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08892ba6-2321-4a83-aa66-eec020295fef None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:49.854908 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 511/1019] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 06:17:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:49.860925 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aaed92-050b-4bbd-a01e-4880d363bc1f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:49.976285 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aaed92-050b-4bbd-a01e-4880d363bc1f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:49.980141 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:50.015460 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3683b-944d-482e-9d5e-b7f3df74ae51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:50.016131 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3683b-944d-482e-9d5e-b7f3df74ae51 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:50.016404 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3683b-944d-482e-9d5e-b7f3df74ae51 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:50.016692 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3683b-944d-482e-9d5e-b7f3df74ae51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:50.017548 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3683b-944d-482e-9d5e-b7f3df74ae51 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:50.018968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3683b-944d-482e-9d5e-b7f3df74ae51 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:50.040126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 509/1020] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 06:17:49 2022] POST /identity/v3/endpoints => generated 334 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:17:51.937948 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd252beb-d9f3-4a22-bd0c-cc65543ed578 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:51.938470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd252beb-d9f3-4a22-bd0c-cc65543ed578 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:51.938550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd252beb-d9f3-4a22-bd0c-cc65543ed578 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:51.940037 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 512/1021] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:51.945409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81d409-b3ad-47ce-90d5-ca62ebf34427 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:51.945802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81d409-b3ad-47ce-90d5-ca62ebf34427 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:51.945972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81d409-b3ad-47ce-90d5-ca62ebf34427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:51.964033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9b81d409-b3ad-47ce-90d5-ca62ebf34427 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:51.965226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9b81d409-b3ad-47ce-90d5-ca62ebf34427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:52.013121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399cbba-7f02-4dac-a390-4f95dc8c54a5 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:52.013444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399cbba-7f02-4dac-a390-4f95dc8c54a5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:52.013726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399cbba-7f02-4dac-a390-4f95dc8c54a5 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:52.015245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 513/1022] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:17:52.020045 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc049b-54d1-4f30-a152-0620bcf7efd6 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:52.020332 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc049b-54d1-4f30-a152-0620bcf7efd6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:52.020610 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc049b-54d1-4f30-a152-0620bcf7efd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:52.039137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-31dc049b-54d1-4f30-a152-0620bcf7efd6 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:52.040877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-31dc049b-54d1-4f30-a152-0620bcf7efd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:52.045871 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 510/1023] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:52.071299 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81d409-b3ad-47ce-90d5-ca62ebf34427 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:52.104089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:52.117142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 514/1024] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:52.139561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc049b-54d1-4f30-a152-0620bcf7efd6 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:52.164038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:52.182927 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81d409-b3ad-47ce-90d5-ca62ebf34427 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:52.222171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be119828-0650-4e54-834a-4eb31a4ca828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:52.223047 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be119828-0650-4e54-834a-4eb31a4ca828 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:52.223339 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be119828-0650-4e54-834a-4eb31a4ca828 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:52.223622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be119828-0650-4e54-834a-4eb31a4ca828 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:52.224833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be119828-0650-4e54-834a-4eb31a4ca828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:52.226310 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be119828-0650-4e54-834a-4eb31a4ca828 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:52.228713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 511/1025] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:52.234331 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be119828-0650-4e54-834a-4eb31a4ca828 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:52.245064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc049b-54d1-4f30-a152-0620bcf7efd6 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:52.283085 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29d259-8309-41e9-a483-615e10cbbc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:52.283633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29d259-8309-41e9-a483-615e10cbbc14 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:52.283944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29d259-8309-41e9-a483-615e10cbbc14 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:52.284229 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29d259-8309-41e9-a483-615e10cbbc14 None admin] PATH_INFO: `/v3/services/image` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:52.284829 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29d259-8309-41e9-a483-615e10cbbc14 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:52.286231 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29d259-8309-41e9-a483-615e10cbbc14 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:52.298148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-6c29d259-8309-41e9-a483-615e10cbbc14 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 06:17:52.298148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:52.298148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:52.298148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:52.298148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:52.298148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:52.298148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:52.298148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:52.298148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 06:17:52.298148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:52.298148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:52.298148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:52.298148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:52.298148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:52.298148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:52.298148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:52.298148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:52.299218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:52.299218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:52.299218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:52.299218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:52.299218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:52.299218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:52.299218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:52.299218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:52.299218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:52.299218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:52.299218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:52.299218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:52.299218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:52.299218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:52.299218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:52.299218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:52.300073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:52.300073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:52.300073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:52.300073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:52.300073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:52.300073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:52.300073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:52.300073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:52.300073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:52.300073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:52.300073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:52.300073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:52.300073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 06:17:52.300073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:52.300073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 515/1026] 10.222.0.65 () {56 vars in 1105 bytes} [Thu May 19 06:17:52 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:52.305664 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29d259-8309-41e9-a483-615e10cbbc14 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:52.359414 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be119828-0650-4e54-834a-4eb31a4ca828 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:52.376290 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:52.401918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70047624-357f-4510-ae02-231650a56421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:52.402746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70047624-357f-4510-ae02-231650a56421 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:52.403033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70047624-357f-4510-ae02-231650a56421 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:52.403940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70047624-357f-4510-ae02-231650a56421 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:52.404974 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70047624-357f-4510-ae02-231650a56421 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:52.406881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70047624-357f-4510-ae02-231650a56421 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:52.423343 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29d259-8309-41e9-a483-615e10cbbc14 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:52.433495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 512/1027] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:17:52 2022] POST /identity/v3/users => generated 285 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:17:52.471465 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea4637-d16e-4de5-a156-638d8bf2e232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:52.472151 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea4637-d16e-4de5-a156-638d8bf2e232 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:52.472500 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea4637-d16e-4de5-a156-638d8bf2e232 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:52.472718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea4637-d16e-4de5-a156-638d8bf2e232 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:52.473477 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea4637-d16e-4de5-a156-638d8bf2e232 None admin] RBAC: Adding query filter params (name=image) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:52.473788 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea4637-d16e-4de5-a156-638d8bf2e232 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:52.475216 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea4637-d16e-4de5-a156-638d8bf2e232 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:52.486336 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 516/1028] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:52.487461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70047624-357f-4510-ae02-231650a56421 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:52.503353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:52.513185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709b056-b2a2-4b7c-9bd5-db2b540314f0 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:52.513532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709b056-b2a2-4b7c-9bd5-db2b540314f0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:52.513817 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709b056-b2a2-4b7c-9bd5-db2b540314f0 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:52.515451 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 517/1029] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:52.521110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2df484-a8fc-4f5e-8771-6d23ad1bc500 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:52.521805 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2df484-a8fc-4f5e-8771-6d23ad1bc500 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:52.521805 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2df484-a8fc-4f5e-8771-6d23ad1bc500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:52.542190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4b2df484-a8fc-4f5e-8771-6d23ad1bc500 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:52.543442 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4b2df484-a8fc-4f5e-8771-6d23ad1bc500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:52.611589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70047624-357f-4510-ae02-231650a56421 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:52.622146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 518/1030] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:52.646120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2df484-a8fc-4f5e-8771-6d23ad1bc500 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:52.655561 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b21ffe-ba6e-4fbd-b3cb-11eddb1fbe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:52.656253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b21ffe-ba6e-4fbd-b3cb-11eddb1fbe35 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:52.656535 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b21ffe-ba6e-4fbd-b3cb-11eddb1fbe35 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:52.656808 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b21ffe-ba6e-4fbd-b3cb-11eddb1fbe35 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:52.657566 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b21ffe-ba6e-4fbd-b3cb-11eddb1fbe35 None admin] RBAC: Adding query filter params (type=image) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:52.657874 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b21ffe-ba6e-4fbd-b3cb-11eddb1fbe35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:52.659261 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b21ffe-ba6e-4fbd-b3cb-11eddb1fbe35 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:52.666113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 513/1031] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:52.671782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b21ffe-ba6e-4fbd-b3cb-11eddb1fbe35 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:52.705086 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:52.762938 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2df484-a8fc-4f5e-8771-6d23ad1bc500 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:52.764174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:52.794038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b21ffe-ba6e-4fbd-b3cb-11eddb1fbe35 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:52.798659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737957e-759f-46a7-8a0c-635b0b72e84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:52.799479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737957e-759f-46a7-8a0c-635b0b72e84c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:52.799766 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737957e-759f-46a7-8a0c-635b0b72e84c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:52.800043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737957e-759f-46a7-8a0c-635b0b72e84c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:52.811241 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1737957e-759f-46a7-8a0c-635b0b72e84c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:17:52.819331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-1737957e-759f-46a7-8a0c-635b0b72e84c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 06:17:52.819331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:52.819331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:52.819331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:52.819331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:52.819331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:52.819331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:52.819331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:52.819331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:52.819331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:52.819331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:52.819331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:52.819331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:17:52.819331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:17:52.819331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:17:52.819331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:17:52.820452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:17:52.820452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:17:52.820452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:17:52.820452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:17:52.820452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:52.820452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:52.820452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:52.820452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:52.820452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:52.820452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:52.820452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:52.820452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:52.820452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:52.820452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:52.820452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:52.820452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:52.821525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:52.821525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:52.821525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:52.821525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:52.821525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:17:52.821525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:17:52.821525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:17:52.821525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:17:52.821525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:17:52.821525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:17:52.821525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 06:17:52.821525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:52.821525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 519/1032] 10.222.0.65 () {56 vars in 1120 bytes} [Thu May 19 06:17:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:52.826839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737957e-759f-46a7-8a0c-635b0b72e84c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:52.834280 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2e410-14fa-4e87-a4d7-0fcbbf1e847c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:52.835186 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2e410-14fa-4e87-a4d7-0fcbbf1e847c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:52.835619 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2e410-14fa-4e87-a4d7-0fcbbf1e847c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:52.836165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2e410-14fa-4e87-a4d7-0fcbbf1e847c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:52.837567 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2e410-14fa-4e87-a4d7-0fcbbf1e847c None admin] RBAC: Adding query filter params (interface=public, service_id=a5f9b9b751204583992a9ebbb9fb20b6, region_id=RegionOne) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:52.837887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2e410-14fa-4e87-a4d7-0fcbbf1e847c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:52.839452 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2e410-14fa-4e87-a4d7-0fcbbf1e847c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:52.849690 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 514/1033] 10.222.0.65 () {56 vars in 1251 bytes} [Thu May 19 06:17:52 2022] GET /identity/v3/endpoints?service_id=a5f9b9b751204583992a9ebbb9fb20b6&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:52.850711 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2e410-14fa-4e87-a4d7-0fcbbf1e847c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:52.932925 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737957e-759f-46a7-8a0c-635b0b72e84c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:52.947533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:52.963678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2e410-14fa-4e87-a4d7-0fcbbf1e847c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:52.973694 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebb330-bf78-421e-87a2-701ab2f4c52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:52.974369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebb330-bf78-421e-87a2-701ab2f4c52e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:52.974648 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebb330-bf78-421e-87a2-701ab2f4c52e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:52.974929 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebb330-bf78-421e-87a2-701ab2f4c52e None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:52.975672 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebb330-bf78-421e-87a2-701ab2f4c52e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:52.975977 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebb330-bf78-421e-87a2-701ab2f4c52e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:52.977980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebb330-bf78-421e-87a2-701ab2f4c52e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:52.986716 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 520/1034] 10.222.0.65 () {56 vars in 1115 bytes} [Thu May 19 06:17:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:52.992256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebb330-bf78-421e-87a2-701ab2f4c52e None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:53.006136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245efd9b-7cd0-4eb3-aee2-8a02cef113e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:53.007155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245efd9b-7cd0-4eb3-aee2-8a02cef113e5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:53.007602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245efd9b-7cd0-4eb3-aee2-8a02cef113e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:53.008038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245efd9b-7cd0-4eb3-aee2-8a02cef113e5 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:53.009613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245efd9b-7cd0-4eb3-aee2-8a02cef113e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:53.011323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245efd9b-7cd0-4eb3-aee2-8a02cef113e5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:53.020282 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 515/1035] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 06:17:52 2022] GET /identity/v3/services => generated 1957 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:17:53.120021 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebb330-bf78-421e-87a2-701ab2f4c52e None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:53.158635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9c653-095d-4866-a90b-5481cdcc59c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:53.159448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9c653-095d-4866-a90b-5481cdcc59c8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:53.159729 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9c653-095d-4866-a90b-5481cdcc59c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:53.160001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9c653-095d-4866-a90b-5481cdcc59c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:53.172125 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9c653-095d-4866-a90b-5481cdcc59c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:53.173627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9c653-095d-4866-a90b-5481cdcc59c8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:53.191999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-25f9c653-095d-4866-a90b-5481cdcc59c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:53.191999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:53.191999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:53.191999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:53.191999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:53.191999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:53.191999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:53.191999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:53.191999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:53.191999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:53.191999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:53.191999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:53.191999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:53.191999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:53.191999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:53.193197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:53.193197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:53.193197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:53.193197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:53.193197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:53.193197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:53.193197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:53.193197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:53.193197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:53.193197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:53.193197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:53.193197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:53.193197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:53.193197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:53.193197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:53.193197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:53.193197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:53.194230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:53.194230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:53.194230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:53.194230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:53.194230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 521/1036] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:53.202600 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245efd9b-7cd0-4eb3-aee2-8a02cef113e5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:53.205523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:53.316755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245efd9b-7cd0-4eb3-aee2-8a02cef113e5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:53.358749 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb8c95-6b2b-4c2b-a681-41a7735115e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:53.359427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb8c95-6b2b-4c2b-a681-41a7735115e8 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:53.359708 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb8c95-6b2b-4c2b-a681-41a7735115e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:53.359981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb8c95-6b2b-4c2b-a681-41a7735115e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:53.360713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb8c95-6b2b-4c2b-a681-41a7735115e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:53.361020 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb8c95-6b2b-4c2b-a681-41a7735115e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:53.362433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb8c95-6b2b-4c2b-a681-41a7735115e8 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:53.380290 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 516/1037] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:53.382187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9c653-095d-4866-a90b-5481cdcc59c8 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:53.410723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:53.492386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9c653-095d-4866-a90b-5481cdcc59c8 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:53.530884 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af56270-e46e-4565-8a5e-95be84498326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:53.531675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af56270-e46e-4565-8a5e-95be84498326 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:53.531954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af56270-e46e-4565-8a5e-95be84498326 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:53.532240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af56270-e46e-4565-8a5e-95be84498326 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:53.541349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af56270-e46e-4565-8a5e-95be84498326 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:53.542808 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af56270-e46e-4565-8a5e-95be84498326 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:53.557479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-1af56270-e46e-4565-8a5e-95be84498326 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 06:17:53.557479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:53.557479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:53.557479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:53.557479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:53.557479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:53.557479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:53.557479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:53.557479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:53.557479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:53.557479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:53.557479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:53.557479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:17:53.557479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:17:53.557479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:17:53.557479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:53.558595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:53.559561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:53.559561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:53.559561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:17:53.559561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:17:53.559561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:17:53.559561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:17:53.559561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:17:53.559561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:17:53.559561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 06:17:53.559561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:53.559561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 522/1038] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 06:17:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:53.564525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb8c95-6b2b-4c2b-a681-41a7735115e8 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:53.632524 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:53.690256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb8c95-6b2b-4c2b-a681-41a7735115e8 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:53.755740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433c05d-75cd-491a-a2a8-e220e4316bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:53.756921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433c05d-75cd-491a-a2a8-e220e4316bda None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:53.757461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433c05d-75cd-491a-a2a8-e220e4316bda None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:53.758095 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433c05d-75cd-491a-a2a8-e220e4316bda None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:53.760912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433c05d-75cd-491a-a2a8-e220e4316bda None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:53.761514 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433c05d-75cd-491a-a2a8-e220e4316bda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:53.764297 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433c05d-75cd-491a-a2a8-e220e4316bda None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:53.805069 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 517/1039] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 06:17:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:53.806579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af56270-e46e-4565-8a5e-95be84498326 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:53.912453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:53.932109 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af56270-e46e-4565-8a5e-95be84498326 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:53.970854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b638cc1-13e3-4764-aef5-93667614e152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:53.971532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b638cc1-13e3-4764-aef5-93667614e152 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:53.971783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b638cc1-13e3-4764-aef5-93667614e152 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:53.972064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b638cc1-13e3-4764-aef5-93667614e152 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:54.049071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b638cc1-13e3-4764-aef5-93667614e152 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:54.050563 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b638cc1-13e3-4764-aef5-93667614e152 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:54.135981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:54.155495 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 523/1040] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:17:53 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:17:54.161695 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433c05d-75cd-491a-a2a8-e220e4316bda None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:54.269722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433c05d-75cd-491a-a2a8-e220e4316bda None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:54.295913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:54.311262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67677a75-50ab-4fea-8125-d297aeeb0417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:54.312062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67677a75-50ab-4fea-8125-d297aeeb0417 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:54.312342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67677a75-50ab-4fea-8125-d297aeeb0417 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:54.312615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67677a75-50ab-4fea-8125-d297aeeb0417 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:54.326528 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67677a75-50ab-4fea-8125-d297aeeb0417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:54.330486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67677a75-50ab-4fea-8125-d297aeeb0417 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:54.347654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-67677a75-50ab-4fea-8125-d297aeeb0417 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:54.347654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:54.347654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:54.347654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:54.347654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:54.347654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:54.347654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:54.347654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:54.347654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:54.347654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:54.347654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:54.347654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:54.347654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:54.347654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:54.347654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:54.348620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:54.348620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:54.348620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:54.348620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:54.348620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:54.348620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:54.348620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:54.348620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:54.348620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:54.348620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:54.348620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:54.348620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:54.348620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:54.348620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:54.348620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:54.348620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:54.348620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:54.349591 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:54.349591 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:54.349591 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:54.349591 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:54.349591 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 518/1041] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:54.358580 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b638cc1-13e3-4764-aef5-93667614e152 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:54.463389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b638cc1-13e3-4764-aef5-93667614e152 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:54.499200 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eeaf60-9780-454a-ab47-fbca847b3826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:54.499880 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eeaf60-9780-454a-ab47-fbca847b3826 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:54.500165 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eeaf60-9780-454a-ab47-fbca847b3826 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:54.500444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eeaf60-9780-454a-ab47-fbca847b3826 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:54.501172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eeaf60-9780-454a-ab47-fbca847b3826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:54.501513 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eeaf60-9780-454a-ab47-fbca847b3826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:54.502895 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eeaf60-9780-454a-ab47-fbca847b3826 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:54.519597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 524/1042] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:54.520922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67677a75-50ab-4fea-8125-d297aeeb0417 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:54.623604 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67677a75-50ab-4fea-8125-d297aeeb0417 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:54.659044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d120a73-ce78-4bc5-bac0-b53ba8b3ec9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:54.659877 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d120a73-ce78-4bc5-bac0-b53ba8b3ec9d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:54.660169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d120a73-ce78-4bc5-bac0-b53ba8b3ec9d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:54.660453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d120a73-ce78-4bc5-bac0-b53ba8b3ec9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:54.669460 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d120a73-ce78-4bc5-bac0-b53ba8b3ec9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:54.670964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d120a73-ce78-4bc5-bac0-b53ba8b3ec9d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:54.686322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-6d120a73-ce78-4bc5-bac0-b53ba8b3ec9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:54.686322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:54.686322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:54.686322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:54.686322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:54.686322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:54.686322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:54.686322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:54.686322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:54.686322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:54.686322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:54.686322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:54.686322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:54.686322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:54.686322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:54.687336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:54.687336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:54.687336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:54.687336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:54.687336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:54.687336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:54.687336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:54.687336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:54.687336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:54.687336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:54.687336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:54.687336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:54.687336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:54.687336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:54.687336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:54.687336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:54.687336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:54.693137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:54.693137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:54.693137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:54.693137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:54.693137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:54.693137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:54.693137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:54.693137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:54.693137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 519/1043] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:54.694086 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eeaf60-9780-454a-ab47-fbca847b3826 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:54.742688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:54.800212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eeaf60-9780-454a-ab47-fbca847b3826 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:54.834886 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc547f87-ccc9-437e-b00f-cd4176bd6eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:54.835559 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc547f87-ccc9-437e-b00f-cd4176bd6eff None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:54.835847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc547f87-ccc9-437e-b00f-cd4176bd6eff None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:54.836133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc547f87-ccc9-437e-b00f-cd4176bd6eff None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:54.837588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc547f87-ccc9-437e-b00f-cd4176bd6eff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:54.837899 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc547f87-ccc9-437e-b00f-cd4176bd6eff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:54.839389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc547f87-ccc9-437e-b00f-cd4176bd6eff None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:54.856697 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 525/1044] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:17:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:54.858127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d120a73-ce78-4bc5-bac0-b53ba8b3ec9d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:54.863828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe302e5-efdd-4210-93ab-9a013e700fa5 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:54.864118 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe302e5-efdd-4210-93ab-9a013e700fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:54.864397 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe302e5-efdd-4210-93ab-9a013e700fa5 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:54.865813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 526/1045] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:17:54.872338 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f8239-df4a-49da-b6c4-b7c1efa81b2e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:54.872711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f8239-df4a-49da-b6c4-b7c1efa81b2e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:54.872913 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f8239-df4a-49da-b6c4-b7c1efa81b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:54.892338 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-120f8239-df4a-49da-b6c4-b7c1efa81b2e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:54.893580 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-120f8239-df4a-49da-b6c4-b7c1efa81b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:54.898053 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:54.972342 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 527/1046] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:54.975964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d120a73-ce78-4bc5-bac0-b53ba8b3ec9d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:54.997261 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f8239-df4a-49da-b6c4-b7c1efa81b2e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:55.015239 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0d4d9-2971-4b4a-bad1-6517d92e3a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:55.015900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0d4d9-2971-4b4a-bad1-6517d92e3a4b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:55.016183 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0d4d9-2971-4b4a-bad1-6517d92e3a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:55.016459 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0d4d9-2971-4b4a-bad1-6517d92e3a4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:55.017353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0d4d9-2971-4b4a-bad1-6517d92e3a4b None admin] RBAC: Adding query filter params (role.id=1a4655a3aaf244afb3988418b88d766f, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=2a09f4878eac4d60a9a7dcedba15c05d) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:55.017668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0d4d9-2971-4b4a-bad1-6517d92e3a4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:55.019115 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0d4d9-2971-4b4a-bad1-6517d92e3a4b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:55.026222 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 520/1047] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:17:54 2022] GET /identity/v3/role_assignments?user.id=2a09f4878eac4d60a9a7dcedba15c05d&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=1a4655a3aaf244afb3988418b88d766f => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:55.043398 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:55.120333 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f8239-df4a-49da-b6c4-b7c1efa81b2e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:55.160401 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573be47-ee07-4d7a-a61d-7aa23bd2939e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:55.161250 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573be47-ee07-4d7a-a61d-7aa23bd2939e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:55.161658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573be47-ee07-4d7a-a61d-7aa23bd2939e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:55.161859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573be47-ee07-4d7a-a61d-7aa23bd2939e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:55.175600 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573be47-ee07-4d7a-a61d-7aa23bd2939e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:17:55.183801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-7573be47-ee07-4d7a-a61d-7aa23bd2939e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:17:55.183801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:55.183801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:55.183801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:55.183801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:55.183801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:55.183801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:55.183801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:55.183801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:55.183801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:55.183801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:55.183801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:55.183801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:17:55.183801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:17:55.183801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:17:55.183801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:17:55.184900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:17:55.184900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:17:55.184900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:17:55.184900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:17:55.184900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:55.184900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:55.184900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:55.184900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:55.184900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:55.184900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:55.184900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:55.184900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:55.184900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:55.184900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:55.184900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:55.184900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:55.186287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:55.186287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:55.186287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:55.186287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:55.186287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:17:55.186287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:17:55.186287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:17:55.186287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:17:55.186287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:17:55.186287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:17:55.186287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:17:55.186287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:55.186287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 528/1048] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:17:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:17:55.194604 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0d4d9-2971-4b4a-bad1-6517d92e3a4b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:55.300661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:55.310362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0d4d9-2971-4b4a-bad1-6517d92e3a4b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:55.343936 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47826a70-0deb-4961-9d7a-753d66824c74 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:55.344236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47826a70-0deb-4961-9d7a-753d66824c74 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:55.344505 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47826a70-0deb-4961-9d7a-753d66824c74 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:55.346121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 529/1049] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:17:55.351161 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7e771-f619-4964-a1a6-6773f1c5237c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:55.351452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7e771-f619-4964-a1a6-6773f1c5237c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:55.351733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7e771-f619-4964-a1a6-6773f1c5237c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:55.359036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c2594-5b48-4672-b514-9c3e978a2c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:55.359711 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c2594-5b48-4672-b514-9c3e978a2c0e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:55.359994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c2594-5b48-4672-b514-9c3e978a2c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:55.360270 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c2594-5b48-4672-b514-9c3e978a2c0e None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:55.361024 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c2594-5b48-4672-b514-9c3e978a2c0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:55.361360 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c2594-5b48-4672-b514-9c3e978a2c0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:55.362743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c2594-5b48-4672-b514-9c3e978a2c0e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:55.369101 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-36c7e771-f619-4964-a1a6-6773f1c5237c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:55.370319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-36c7e771-f619-4964-a1a6-6773f1c5237c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:55.376523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 521/1050] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:17:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:55.379032 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c2594-5b48-4672-b514-9c3e978a2c0e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:55.447956 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 530/1051] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:55.470442 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7e771-f619-4964-a1a6-6773f1c5237c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:55.495215 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c2594-5b48-4672-b514-9c3e978a2c0e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:55.497923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:55.537031 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85658053-8b01-447c-a31e-26b64c06581b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:55.538027 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85658053-8b01-447c-a31e-26b64c06581b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:55.538308 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85658053-8b01-447c-a31e-26b64c06581b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:55.538647 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85658053-8b01-447c-a31e-26b64c06581b None admin] PATH_INFO: `/v3/users/b550ec0df73b44ff987929104bec3c36` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:55.556635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85658053-8b01-447c-a31e-26b64c06581b None admin] RBAC: Authorizing `identity:get_user(user_id=b550ec0df73b44ff987929104bec3c36)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:55.558133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85658053-8b01-447c-a31e-26b64c06581b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:55.559889 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 522/1052] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:17:55 2022] GET /identity/v3/users/b550ec0df73b44ff987929104bec3c36 => generated 285 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:55.565558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85658053-8b01-447c-a31e-26b64c06581b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:55.584694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:55.586347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7e771-f619-4964-a1a6-6773f1c5237c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:55.621624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cefb90d-723c-47d5-a334-9dc2cd46079d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:55.622419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefb90d-723c-47d5-a334-9dc2cd46079d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:55.622709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefb90d-723c-47d5-a334-9dc2cd46079d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:55.622988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefb90d-723c-47d5-a334-9dc2cd46079d None admin] PATH_INFO: `/v3/services/image` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:55.623643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cefb90d-723c-47d5-a334-9dc2cd46079d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:55.625024 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cefb90d-723c-47d5-a334-9dc2cd46079d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:55.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-3cefb90d-723c-47d5-a334-9dc2cd46079d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 06:17:55.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:55.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:17:55.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:17:55.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:17:55.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:17:55.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:17:55.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:55.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 06:17:55.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:55.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:17:55.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:55.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:55.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:55.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:55.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:55.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:55.638192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:55.638192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:55.638192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:55.638192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:55.638192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:55.638192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:55.638192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:17:55.638192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:17:55.638192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:17:55.638192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:17:55.638192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:55.638192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:55.638192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:55.638192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:55.638192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:55.638192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:55.639389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:55.639389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:55.639389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:55.639389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:55.639389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:55.639389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:55.639389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:55.639389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:55.639389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:55.639389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:55.639389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:17:55.639389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:17:55.639389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 06:17:55.639389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:55.643235 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 531/1053] 10.222.0.65 () {56 vars in 1105 bytes} [Thu May 19 06:17:55 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:55.644200 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cefb90d-723c-47d5-a334-9dc2cd46079d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:55.687041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85658053-8b01-447c-a31e-26b64c06581b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:55.726228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadec52-d614-4e65-be91-481c81567e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:55.726910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadec52-d614-4e65-be91-481c81567e7a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:55.727245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadec52-d614-4e65-be91-481c81567e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:55.728011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadec52-d614-4e65-be91-481c81567e7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:55.743932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:55.766359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cefb90d-723c-47d5-a334-9dc2cd46079d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:55.804283 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e43898-95ea-42d4-895c-87c20d4319e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:55.804951 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e43898-95ea-42d4-895c-87c20d4319e3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:55.805266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e43898-95ea-42d4-895c-87c20d4319e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:55.805574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e43898-95ea-42d4-895c-87c20d4319e3 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:55.806286 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e43898-95ea-42d4-895c-87c20d4319e3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:55.806405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadec52-d614-4e65-be91-481c81567e7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:55.806587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e43898-95ea-42d4-895c-87c20d4319e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:55.807744 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadec52-d614-4e65-be91-481c81567e7a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:55.807951 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e43898-95ea-42d4-895c-87c20d4319e3 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:55.813217 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 532/1054] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:55.818622 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e43898-95ea-42d4-895c-87c20d4319e3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:55.827826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:55.926986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 523/1055] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:17:55 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 06:17:55.933783 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadec52-d614-4e65-be91-481c81567e7a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:55.937864 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e43898-95ea-42d4-895c-87c20d4319e3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:55.974694 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af98f7-c993-408e-9305-499aad9b8c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:55.975551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af98f7-c993-408e-9305-499aad9b8c29 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:55.975848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af98f7-c993-408e-9305-499aad9b8c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:55.976131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af98f7-c993-408e-9305-499aad9b8c29 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:55.976845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af98f7-c993-408e-9305-499aad9b8c29 None admin] RBAC: Adding query filter params (type=image) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:55.977452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af98f7-c993-408e-9305-499aad9b8c29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:55.978695 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af98f7-c993-408e-9305-499aad9b8c29 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:55.983767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 533/1056] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:55.989453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af98f7-c993-408e-9305-499aad9b8c29 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:56.009865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:56.043430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadec52-d614-4e65-be91-481c81567e7a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:56.067545 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:56.089605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f776c-ea72-4f3e-bce5-a9427c4369b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:56.090477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f776c-ea72-4f3e-bce5-a9427c4369b4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:56.090764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f776c-ea72-4f3e-bce5-a9427c4369b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:56.091045 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f776c-ea72-4f3e-bce5-a9427c4369b4 None admin] PATH_INFO: `/v3/projects/d968876d970b4fba86f096e789b85ab1` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:56.092267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f776c-ea72-4f3e-bce5-a9427c4369b4 None admin] RBAC: Authorizing `identity:get_project(project_id=d968876d970b4fba86f096e789b85ab1)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:56.093740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f776c-ea72-4f3e-bce5-a9427c4369b4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:56.097314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 524/1057] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:17:55 2022] GET /identity/v3/projects/d968876d970b4fba86f096e789b85ab1 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:56.103712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f776c-ea72-4f3e-bce5-a9427c4369b4 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:56.109401 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af98f7-c993-408e-9305-499aad9b8c29 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:56.167845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985e439-9bc2-4cbc-aebf-d54c84e549b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:56.168520 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985e439-9bc2-4cbc-aebf-d54c84e549b6 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:56.168812 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985e439-9bc2-4cbc-aebf-d54c84e549b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:56.169089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985e439-9bc2-4cbc-aebf-d54c84e549b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:56.169938 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985e439-9bc2-4cbc-aebf-d54c84e549b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:56.171497 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985e439-9bc2-4cbc-aebf-d54c84e549b6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:56.205421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 534/1058] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 06:17:55 2022] POST /identity/v3/endpoints => generated 327 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:17:56.224044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f776c-ea72-4f3e-bce5-a9427c4369b4 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:56.277058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43f03f-7f93-46e3-837e-0d3cd2afc7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:56.277772 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43f03f-7f93-46e3-837e-0d3cd2afc7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:56.278055 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43f03f-7f93-46e3-837e-0d3cd2afc7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:56.278333 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43f03f-7f93-46e3-837e-0d3cd2afc7a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:56.279501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43f03f-7f93-46e3-837e-0d3cd2afc7a7 None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=d968876d970b4fba86f096e789b85ab1, user.id=b550ec0df73b44ff987929104bec3c36) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:56.279819 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43f03f-7f93-46e3-837e-0d3cd2afc7a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:56.281417 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43f03f-7f93-46e3-837e-0d3cd2afc7a7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:56.293326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 525/1059] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:17:56 2022] GET /identity/v3/role_assignments?user.id=b550ec0df73b44ff987929104bec3c36&scope.project.id=d968876d970b4fba86f096e789b85ab1&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:57.325561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa13a3e-4193-4e93-8976-a87f89bea89d None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:57.326082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa13a3e-4193-4e93-8976-a87f89bea89d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:57.326188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa13a3e-4193-4e93-8976-a87f89bea89d None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:57.327663 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 535/1060] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:57.334200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0aa5d-8e09-460a-b81b-7549f5a6e2be None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:57.334200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0aa5d-8e09-460a-b81b-7549f5a6e2be None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:57.334606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0aa5d-8e09-460a-b81b-7549f5a6e2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:57.353757 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-7aa0aa5d-8e09-460a-b81b-7549f5a6e2be None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:57.354967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7aa0aa5d-8e09-460a-b81b-7549f5a6e2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:57.403740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:57.437210 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 526/1061] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:17:57.460576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa13a3e-4193-4e93-8976-a87f89bea89d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:57.542333 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:57.582141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa13a3e-4193-4e93-8976-a87f89bea89d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:57.618523 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72c460-f396-47c3-9b95-c8f3d643c161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:57.619339 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72c460-f396-47c3-9b95-c8f3d643c161 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:57.619626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72c460-f396-47c3-9b95-c8f3d643c161 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:57.619902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72c460-f396-47c3-9b95-c8f3d643c161 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:57.630246 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72c460-f396-47c3-9b95-c8f3d643c161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:17:57.638413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-5c72c460-f396-47c3-9b95-c8f3d643c161 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 06:17:57.638413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:57.638413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:57.638413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:57.638413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:57.638413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:57.638413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:57.638413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:57.638413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:57.638413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:57.638413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:57.638413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:57.638413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:17:57.638413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:17:57.638413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:17:57.638413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:17:57.639349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:17:57.639349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:17:57.639349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:17:57.639349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:17:57.639349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:57.639349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:57.639349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:57.639349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:57.639349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:57.639349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:57.639349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:57.639349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:57.639349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:57.639349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:57.639349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:57.639349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:57.640191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:57.640191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:57.640191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:57.640191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:57.640191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:17:57.640191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:17:57.640191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:17:57.640191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:17:57.640191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:17:57.640191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:17:57.640191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 06:17:57.640191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:57.647758 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 536/1062] 10.222.0.65 () {56 vars in 1120 bytes} [Thu May 19 06:17:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:57.647906 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0aa5d-8e09-460a-b81b-7549f5a6e2be None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:57.750973 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0aa5d-8e09-460a-b81b-7549f5a6e2be None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:57.786336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbee94-bc37-492d-a7ff-e5b186d299be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:57.787012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbee94-bc37-492d-a7ff-e5b186d299be None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:57.787304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbee94-bc37-492d-a7ff-e5b186d299be None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:57.787584 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbee94-bc37-492d-a7ff-e5b186d299be None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:57.788339 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbee94-bc37-492d-a7ff-e5b186d299be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:57.788647 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbee94-bc37-492d-a7ff-e5b186d299be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:57.790638 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbee94-bc37-492d-a7ff-e5b186d299be None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:57.803606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 527/1063] 10.222.0.65 () {56 vars in 1115 bytes} [Thu May 19 06:17:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:57.804748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72c460-f396-47c3-9b95-c8f3d643c161 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:57.909574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72c460-f396-47c3-9b95-c8f3d643c161 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:57.939358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:57.945926 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fe92a-a3fa-440b-ab66-c2c2bccfd8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:57.946739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fe92a-a3fa-440b-ab66-c2c2bccfd8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:57.947025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fe92a-a3fa-440b-ab66-c2c2bccfd8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:57.947303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fe92a-a3fa-440b-ab66-c2c2bccfd8b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:57.958588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fe92a-a3fa-440b-ab66-c2c2bccfd8b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:57.960004 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fe92a-a3fa-440b-ab66-c2c2bccfd8b9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:57.979102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-733fe92a-a3fa-440b-ab66-c2c2bccfd8b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:57.979102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:57.979102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:57.979102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:57.979102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:57.979102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:57.979102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:57.979102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:57.979102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:57.979102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:57.979102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:57.979102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:57.979102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:57.979102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:57.979102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:57.980064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:57.980064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:57.980064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:57.980064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:57.980064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:57.980064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:57.980064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:57.980064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:57.980064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:57.980064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:57.980064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:57.980064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:57.980064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:57.980064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:57.980064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:57.980064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:57.980064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:57.981073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:57.981073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:57.981073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:57.981073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:57.981073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 537/1064] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:57.987171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbee94-bc37-492d-a7ff-e5b186d299be None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:58.026854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:58.098492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbee94-bc37-492d-a7ff-e5b186d299be None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:58.133493 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0f2f9-110f-47c3-84bb-a48b8e8268c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:58.134296 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0f2f9-110f-47c3-84bb-a48b8e8268c5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:58.134501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0f2f9-110f-47c3-84bb-a48b8e8268c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:58.134729 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0f2f9-110f-47c3-84bb-a48b8e8268c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:58.135460 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0f2f9-110f-47c3-84bb-a48b8e8268c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:58.135767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0f2f9-110f-47c3-84bb-a48b8e8268c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:58.137154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0f2f9-110f-47c3-84bb-a48b8e8268c5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:58.155043 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 528/1065] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:58.156384 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fe92a-a3fa-440b-ab66-c2c2bccfd8b9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:58.261372 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fe92a-a3fa-440b-ab66-c2c2bccfd8b9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:58.295722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106302f5-00c2-4eb4-bcd4-ad59ccdbd3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:58.296515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106302f5-00c2-4eb4-bcd4-ad59ccdbd3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:58.296800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106302f5-00c2-4eb4-bcd4-ad59ccdbd3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:58.297077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106302f5-00c2-4eb4-bcd4-ad59ccdbd3e9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:58.304638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106302f5-00c2-4eb4-bcd4-ad59ccdbd3e9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:58.307011 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106302f5-00c2-4eb4-bcd4-ad59ccdbd3e9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:58.321605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-106302f5-00c2-4eb4-bcd4-ad59ccdbd3e9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 06:17:58.321605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:58.321605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:58.321605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:58.321605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:58.321605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:58.321605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:58.321605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:58.321605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:58.321605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:58.321605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:58.321605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:58.321605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:17:58.321605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:17:58.321605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:17:58.321605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:58.322641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:58.323638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:58.323638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:58.323638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:17:58.323638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:17:58.323638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:17:58.323638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:17:58.323638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:17:58.323638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:17:58.323638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 06:17:58.323638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:58.323638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 538/1066] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 06:17:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:58.331984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0f2f9-110f-47c3-84bb-a48b8e8268c5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:58.441575 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:58.449796 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0f2f9-110f-47c3-84bb-a48b8e8268c5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:58.485058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345236f0-0479-49e2-8506-7097ed728728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:58.485778 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345236f0-0479-49e2-8506-7097ed728728 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:58.486066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345236f0-0479-49e2-8506-7097ed728728 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:58.486346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345236f0-0479-49e2-8506-7097ed728728 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:58.487192 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345236f0-0479-49e2-8506-7097ed728728 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:58.487501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345236f0-0479-49e2-8506-7097ed728728 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:58.488933 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345236f0-0479-49e2-8506-7097ed728728 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:58.511974 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 529/1067] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 06:17:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:17:58.513207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106302f5-00c2-4eb4-bcd4-ad59ccdbd3e9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:58.522088 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe90e6-4572-4b16-a6fb-c1c67c567a51 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:58.522376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe90e6-4572-4b16-a6fb-c1c67c567a51 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:58.522656 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe90e6-4572-4b16-a6fb-c1c67c567a51 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:58.524769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 530/1068] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:17:58.530739 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503b85d-ad5c-4e7f-b302-d9f4e9df5431 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:58.531026 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503b85d-ad5c-4e7f-b302-d9f4e9df5431 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:58.531403 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503b85d-ad5c-4e7f-b302-d9f4e9df5431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:58.547789 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d503b85d-ad5c-4e7f-b302-d9f4e9df5431 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:58.548960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d503b85d-ad5c-4e7f-b302-d9f4e9df5431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:58.569452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:58.620155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 531/1069] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:58.621539 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89587d2-552d-4c74-9ff9-a906c69ec5f7 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:58.621833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89587d2-552d-4c74-9ff9-a906c69ec5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:58.622156 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89587d2-552d-4c74-9ff9-a906c69ec5f7 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:58.625330 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 532/1070] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:17:58.626314 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106302f5-00c2-4eb4-bcd4-ad59ccdbd3e9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:58.630782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74c870-c731-4238-b2e7-924af015731b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:58.631074 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74c870-c731-4238-b2e7-924af015731b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:58.631355 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74c870-c731-4238-b2e7-924af015731b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:58.650737 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-fd74c870-c731-4238-b2e7-924af015731b None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:17:58.651941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fd74c870-c731-4238-b2e7-924af015731b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:17:58.661807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf5116-c0de-4a89-ba2f-71281f40bd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:58.662608 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf5116-c0de-4a89-ba2f-71281f40bd9b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:58.662894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf5116-c0de-4a89-ba2f-71281f40bd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:58.663174 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf5116-c0de-4a89-ba2f-71281f40bd9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:58.674400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf5116-c0de-4a89-ba2f-71281f40bd9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:58.675860 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf5116-c0de-4a89-ba2f-71281f40bd9b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:58.692603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-b5cf5116-c0de-4a89-ba2f-71281f40bd9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:58.692603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:58.692603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:58.692603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:58.692603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:58.692603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:58.692603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:58.692603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:58.692603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:58.692603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:58.692603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:58.692603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:58.692603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:58.692603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:58.692603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:58.693709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:58.693709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:58.693709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:58.693709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:58.693709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:58.693709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:58.693709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:58.693709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:58.693709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:58.693709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:58.693709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:58.693709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:58.693709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:58.693709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:58.693709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:58.693709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:58.693709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:58.694633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:58.694633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:58.694633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:58.694633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:58.694633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 539/1071] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:58.694633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf5116-c0de-4a89-ba2f-71281f40bd9b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:58.724271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 533/1072] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:17:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:17:58.725853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74c870-c731-4238-b2e7-924af015731b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:58.741483 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:58.807953 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:58.814400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf5116-c0de-4a89-ba2f-71281f40bd9b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:58.830456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74c870-c731-4238-b2e7-924af015731b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:58.848500 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce8db8-3b29-47d4-9647-ebf289e9d7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:58.849158 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce8db8-3b29-47d4-9647-ebf289e9d7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:58.849479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce8db8-3b29-47d4-9647-ebf289e9d7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:58.849762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce8db8-3b29-47d4-9647-ebf289e9d7d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:58.864871 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ae503-5039-4584-ae19-e510e4223f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:58.865565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ae503-5039-4584-ae19-e510e4223f3f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:58.865846 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ae503-5039-4584-ae19-e510e4223f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:58.866120 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ae503-5039-4584-ae19-e510e4223f3f None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:58.866838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ae503-5039-4584-ae19-e510e4223f3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:58.867144 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ae503-5039-4584-ae19-e510e4223f3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:58.868541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ae503-5039-4584-ae19-e510e4223f3f None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:58.879652 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 534/1073] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:58.880438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ae503-5039-4584-ae19-e510e4223f3f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:58.923968 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce8db8-3b29-47d4-9647-ebf289e9d7d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:58.925467 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce8db8-3b29-47d4-9647-ebf289e9d7d0 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:58.955942 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:58.982533 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ae503-5039-4584-ae19-e510e4223f3f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:59.018272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cec9b6-e64f-4287-a9c8-dda1627a7c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:59.019160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cec9b6-e64f-4287-a9c8-dda1627a7c31 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:59.019548 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cec9b6-e64f-4287-a9c8-dda1627a7c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:59.019679 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:59.019786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cec9b6-e64f-4287-a9c8-dda1627a7c31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:59.026997 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 540/1074] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:17:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:17:59.027810 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce8db8-3b29-47d4-9647-ebf289e9d7d0 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:59.031810 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cec9b6-e64f-4287-a9c8-dda1627a7c31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:17:59.039613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-e4cec9b6-e64f-4287-a9c8-dda1627a7c31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:17:59.039613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:59.039613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:59.039613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:59.039613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:59.039613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:59.039613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:59.039613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:59.039613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:59.039613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:59.039613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:59.039613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:59.039613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:17:59.039613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:17:59.039613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:17:59.039613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:17:59.040937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:17:59.040937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:17:59.040937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:17:59.040937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:17:59.040937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:59.040937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:59.040937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:59.040937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:59.040937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:59.040937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:59.040937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:59.040937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:59.040937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:59.040937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:59.040937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:59.040937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:59.042011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:59.042011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:59.042011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:59.042011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:59.042011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:17:59.042011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:17:59.042011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:17:59.042011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:17:59.042011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:17:59.042011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:17:59.042011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:17:59.042011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:59.042011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 535/1075] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:17:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:59.042011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cec9b6-e64f-4287-a9c8-dda1627a7c31 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:59.138412 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce8db8-3b29-47d4-9647-ebf289e9d7d0 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:59.150386 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cec9b6-e64f-4287-a9c8-dda1627a7c31 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:59.174582 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5106a9b-289f-4b2c-aacb-10075f9264dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:59.175380 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5106a9b-289f-4b2c-aacb-10075f9264dc None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:59.175663 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5106a9b-289f-4b2c-aacb-10075f9264dc None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:59.175942 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5106a9b-289f-4b2c-aacb-10075f9264dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:59.182903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5106a9b-289f-4b2c-aacb-10075f9264dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:59.184348 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5106a9b-289f-4b2c-aacb-10075f9264dc None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:59.184729 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7baa89-bfa2-4fd4-af83-2c940e58a427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:59.185543 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7baa89-bfa2-4fd4-af83-2c940e58a427 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:59.185826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7baa89-bfa2-4fd4-af83-2c940e58a427 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:59.186108 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7baa89-bfa2-4fd4-af83-2c940e58a427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:59.188372 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:59.198886 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7baa89-bfa2-4fd4-af83-2c940e58a427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:59.199675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-d5106a9b-289f-4b2c-aacb-10075f9264dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:59.199675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:59.199675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:59.199675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:59.199675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:59.199675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:59.199675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:59.199675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:59.199675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:59.199675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:59.199675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:59.199675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:59.199675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:17:59.199675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:17:59.199675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:17:59.200570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:17:59.200570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:59.200570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:59.200570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:59.200570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:59.200570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:59.200570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:59.200570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:59.200570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:17:59.200570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:59.200570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:59.200570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:59.200570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:59.200570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:59.200570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:59.200570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:59.200570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:59.201522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:17:59.201522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:17:59.201522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:17:59.201522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:17:59.201522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:17:59.201522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:17:59.201522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:17:59.201522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:17:59.201960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7baa89-bfa2-4fd4-af83-2c940e58a427 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:59.202038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 541/1076] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:17:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:59.202038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5106a9b-289f-4b2c-aacb-10075f9264dc None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:59.220918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-4c7baa89-bfa2-4fd4-af83-2c940e58a427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:59.220918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:17:59.220918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:17:59.220918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:17:59.220918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:17:59.220918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:17:59.220918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:17:59.220918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:17:59.220918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:17:59.220918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:17:59.220918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:17:59.220918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:17:59.220918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:17:59.220918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:17:59.220918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:17:59.222072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:17:59.222072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:17:59.222072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:17:59.222072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:17:59.222072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:17:59.222072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:17:59.222072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:17:59.222072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:17:59.222072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:17:59.222072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:17:59.222072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:17:59.222072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:17:59.222072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:17:59.222072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:17:59.222072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:17:59.222072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:17:59.222072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:17:59.223006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:17:59.223006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:17:59.223006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:17:59.223006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:17:59.223548 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 536/1077] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:17:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:17:59.224136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7baa89-bfa2-4fd4-af83-2c940e58a427 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:59.259142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:59.305738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5106a9b-289f-4b2c-aacb-10075f9264dc None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:59.326463 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7baa89-bfa2-4fd4-af83-2c940e58a427 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:59.343352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13aa2c-777e-47a1-adc8-5bbb5541345c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:59.344031 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13aa2c-777e-47a1-adc8-5bbb5541345c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:59.344314 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13aa2c-777e-47a1-adc8-5bbb5541345c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:59.344594 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13aa2c-777e-47a1-adc8-5bbb5541345c None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:59.345369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13aa2c-777e-47a1-adc8-5bbb5541345c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:59.345676 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13aa2c-777e-47a1-adc8-5bbb5541345c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:59.347054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13aa2c-777e-47a1-adc8-5bbb5541345c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:59.355393 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 542/1078] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:17:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:59.356279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13aa2c-777e-47a1-adc8-5bbb5541345c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:59.364320 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62931fcc-5214-4bb3-81ee-15cd12c149cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:59.365471 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62931fcc-5214-4bb3-81ee-15cd12c149cd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:59.365766 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62931fcc-5214-4bb3-81ee-15cd12c149cd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:59.366048 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62931fcc-5214-4bb3-81ee-15cd12c149cd None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:59.367207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62931fcc-5214-4bb3-81ee-15cd12c149cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:59.367549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62931fcc-5214-4bb3-81ee-15cd12c149cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:59.369329 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62931fcc-5214-4bb3-81ee-15cd12c149cd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:59.381879 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 537/1079] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:17:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:59.382695 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62931fcc-5214-4bb3-81ee-15cd12c149cd None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:59.436219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:59.463819 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13aa2c-777e-47a1-adc8-5bbb5541345c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:59.482361 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:59.495043 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62931fcc-5214-4bb3-81ee-15cd12c149cd None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:59.500436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808f70-6399-472b-8c15-3291fa2d6afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:59.501101 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808f70-6399-472b-8c15-3291fa2d6afd None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:59.501419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808f70-6399-472b-8c15-3291fa2d6afd None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:59.501696 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808f70-6399-472b-8c15-3291fa2d6afd None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:59.502414 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808f70-6399-472b-8c15-3291fa2d6afd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:17:59.502721 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808f70-6399-472b-8c15-3291fa2d6afd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:59.504101 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808f70-6399-472b-8c15-3291fa2d6afd None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:59.517148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 543/1080] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:59.517974 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808f70-6399-472b-8c15-3291fa2d6afd None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:59.535130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ba4d8-7163-4e2c-bdc1-f5fab6829cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:59.536809 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ba4d8-7163-4e2c-bdc1-f5fab6829cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:59.537348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ba4d8-7163-4e2c-bdc1-f5fab6829cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:59.537882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ba4d8-7163-4e2c-bdc1-f5fab6829cf8 None admin] PATH_INFO: `/v3/users/b550ec0df73b44ff987929104bec3c36` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:59.540873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ba4d8-7163-4e2c-bdc1-f5fab6829cf8 None admin] RBAC: Authorizing `identity:get_user(user_id=b550ec0df73b44ff987929104bec3c36)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:59.543971 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ba4d8-7163-4e2c-bdc1-f5fab6829cf8 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:59.551403 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 538/1081] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:17:59 2022] GET /identity/v3/users/b550ec0df73b44ff987929104bec3c36 => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:59.553119 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ba4d8-7163-4e2c-bdc1-f5fab6829cf8 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:59.641106 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808f70-6399-472b-8c15-3291fa2d6afd None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:59.685950 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94add91-781a-496e-94dc-09814ef1de47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:59.686932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94add91-781a-496e-94dc-09814ef1de47 None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:59.687158 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94add91-781a-496e-94dc-09814ef1de47 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:59.687439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94add91-781a-496e-94dc-09814ef1de47 None admin] PATH_INFO: `/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/2a09f4878eac4d60a9a7dcedba15c05d/roles/1a4655a3aaf244afb3988418b88d766f` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:59.688448 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ba4d8-7163-4e2c-bdc1-f5fab6829cf8 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:59.690041 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94add91-781a-496e-94dc-09814ef1de47 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3d399dc95424b848b736c0d0298d9a9, user_id=2a09f4878eac4d60a9a7dcedba15c05d, role_id=1a4655a3aaf244afb3988418b88d766f)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:59.691667 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94add91-781a-496e-94dc-09814ef1de47 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:59.707486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:59.717322 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 544/1082] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:17:59 2022] PUT /identity/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/2a09f4878eac4d60a9a7dcedba15c05d/roles/1a4655a3aaf244afb3988418b88d766f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:17:59.746313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aface868-14bc-4685-9c10-e2fbd86548fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:59.747229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aface868-14bc-4685-9c10-e2fbd86548fa None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:59.747517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aface868-14bc-4685-9c10-e2fbd86548fa None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:59.747798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aface868-14bc-4685-9c10-e2fbd86548fa None admin] PATH_INFO: `/v3/projects/d968876d970b4fba86f096e789b85ab1` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:59.749146 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aface868-14bc-4685-9c10-e2fbd86548fa None admin] RBAC: Authorizing `identity:get_project(project_id=d968876d970b4fba86f096e789b85ab1)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:59.750685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aface868-14bc-4685-9c10-e2fbd86548fa None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:59.752550 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 539/1083] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:17:59 2022] GET /identity/v3/projects/d968876d970b4fba86f096e789b85ab1 => generated 311 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:17:59.759217 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94add91-781a-496e-94dc-09814ef1de47 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:17:59.812748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:17:59.913703 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94add91-781a-496e-94dc-09814ef1de47 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:17:59.950025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42c40c-d4dc-40fe-ae6b-109489e4699b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:17:59.950829 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42c40c-d4dc-40fe-ae6b-109489e4699b None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:17:59.951110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42c40c-d4dc-40fe-ae6b-109489e4699b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:17:59.951383 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42c40c-d4dc-40fe-ae6b-109489e4699b None admin] PATH_INFO: `/v3/projects/d968876d970b4fba86f096e789b85ab1/users/b550ec0df73b44ff987929104bec3c36/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:17:59.953366 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42c40c-d4dc-40fe-ae6b-109489e4699b None admin] RBAC: Authorizing `identity:create_grant(project_id=d968876d970b4fba86f096e789b85ab1, user_id=b550ec0df73b44ff987929104bec3c36, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:17:59.954811 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42c40c-d4dc-40fe-ae6b-109489e4699b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:17:59.969332 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 545/1084] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:17:59 2022] PUT /identity/v3/projects/d968876d970b4fba86f096e789b85ab1/users/b550ec0df73b44ff987929104bec3c36/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:18:01.986714 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca8717-46fc-4bd8-b9a3-cc2e111ff367 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:01.987276 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca8717-46fc-4bd8-b9a3-cc2e111ff367 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:01.987276 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca8717-46fc-4bd8-b9a3-cc2e111ff367 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:01.988644 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 540/1085] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:01.994859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8833c-18b5-4c8d-b2e8-e0cddbabb522 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:01.995273 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8833c-18b5-4c8d-b2e8-e0cddbabb522 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:01.995424 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8833c-18b5-4c8d-b2e8-e0cddbabb522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:02.012831 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e1b8833c-18b5-4c8d-b2e8-e0cddbabb522 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:02.014204 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e1b8833c-18b5-4c8d-b2e8-e0cddbabb522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:02.087282 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 546/1086] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:18:02.111426 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca8717-46fc-4bd8-b9a3-cc2e111ff367 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:02.212465 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781a221-2ce8-4e61-8a46-6749796fcbc4 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:02.212762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781a221-2ce8-4e61-8a46-6749796fcbc4 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:02.213050 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781a221-2ce8-4e61-8a46-6749796fcbc4 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:02.214459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 547/1087] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:02.218507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca8717-46fc-4bd8-b9a3-cc2e111ff367 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:02.219586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4d875-5076-47d4-b45b-b4e64bbf8d49 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:02.219878 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4d875-5076-47d4-b45b-b4e64bbf8d49 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:02.220163 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4d875-5076-47d4-b45b-b4e64bbf8d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:02.239133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-76b4d875-5076-47d4-b45b-b4e64bbf8d49 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:02.240267 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-76b4d875-5076-47d4-b45b-b4e64bbf8d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:02.254074 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3d6d0-f756-4504-b431-6fcb98ad378e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:02.254882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3d6d0-f756-4504-b431-6fcb98ad378e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:02.255161 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3d6d0-f756-4504-b431-6fcb98ad378e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:02.255432 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3d6d0-f756-4504-b431-6fcb98ad378e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:02.265023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:02.265667 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3d6d0-f756-4504-b431-6fcb98ad378e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:18:02.273782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f9d3d6d0-f756-4504-b431-6fcb98ad378e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 06:18:02.273782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:18:02.273782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:18:02.273782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:18:02.273782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:18:02.273782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:18:02.273782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:18:02.273782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:18:02.273782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:18:02.273782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:18:02.273782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:18:02.273782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:18:02.273782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:18:02.273782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:18:02.273782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:18:02.273782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:18:02.274949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:18:02.274949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:18:02.274949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:18:02.274949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:18:02.274949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:18:02.274949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:18:02.274949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:18:02.274949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:18:02.274949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:18:02.274949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:18:02.274949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:18:02.274949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:18:02.274949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:18:02.274949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:18:02.274949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:18:02.274949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:18:02.275824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:18:02.275824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:18:02.275824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:18:02.275824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:18:02.275824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:18:02.275824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:18:02.275824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:18:02.275824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:18:02.275824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:18:02.275824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:18:02.275824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 06:18:02.275824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:18:02.275824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 541/1088] 10.222.0.65 () {56 vars in 1120 bytes} [Thu May 19 06:18:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:18:02.281164 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3d6d0-f756-4504-b431-6fcb98ad378e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:02.312831 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 548/1089] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:18:02.325831 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:02.337087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4d875-5076-47d4-b45b-b4e64bbf8d49 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:02.386701 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3d6d0-f756-4504-b431-6fcb98ad378e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:02.421651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d3d6a-9eb6-4282-bb6a-658edfa45bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:02.422316 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d3d6a-9eb6-4282-bb6a-658edfa45bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:02.422601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d3d6a-9eb6-4282-bb6a-658edfa45bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:02.423051 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d3d6a-9eb6-4282-bb6a-658edfa45bf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:02.423703 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d3d6a-9eb6-4282-bb6a-658edfa45bf5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:02.424071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d3d6a-9eb6-4282-bb6a-658edfa45bf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:02.425489 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d3d6a-9eb6-4282-bb6a-658edfa45bf5 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:02.434101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 542/1090] 10.222.0.65 () {56 vars in 1115 bytes} [Thu May 19 06:18:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:18:02.439746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d3d6a-9eb6-4282-bb6a-658edfa45bf5 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:02.440585 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4d875-5076-47d4-b45b-b4e64bbf8d49 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:02.476599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488ce72-0540-4aea-917b-00d0043d1856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:02.477982 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488ce72-0540-4aea-917b-00d0043d1856 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:02.478271 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488ce72-0540-4aea-917b-00d0043d1856 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:02.478547 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488ce72-0540-4aea-917b-00d0043d1856 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:02.488931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488ce72-0540-4aea-917b-00d0043d1856 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:18:02.496743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-7488ce72-0540-4aea-917b-00d0043d1856 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:18:02.496743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:18:02.496743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:18:02.496743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:18:02.496743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:18:02.496743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:18:02.496743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:18:02.496743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:18:02.496743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:18:02.496743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:18:02.496743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:18:02.496743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:18:02.496743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:18:02.496743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:18:02.496743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:18:02.496743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:18:02.498281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:18:02.498281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:18:02.498281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:18:02.498281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:18:02.498281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:18:02.498281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:18:02.498281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:18:02.498281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:18:02.498281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:18:02.498281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:18:02.498281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:18:02.498281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:18:02.498281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:18:02.498281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:18:02.498281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:18:02.498281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:18:02.499447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:18:02.499447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:18:02.499447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:18:02.499447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:18:02.499447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:18:02.499447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:18:02.499447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:18:02.499447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:18:02.499447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:18:02.499447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:18:02.499447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:18:02.499447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:18:02.499447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 549/1091] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:18:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:18:02.503944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488ce72-0540-4aea-917b-00d0043d1856 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:02.524464 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:02.554668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d3d6a-9eb6-4282-bb6a-658edfa45bf5 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:02.557418 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:02.589329 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3ea91-bb3b-489a-a491-04192111cee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:02.590131 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3ea91-bb3b-489a-a491-04192111cee3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:02.590417 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3ea91-bb3b-489a-a491-04192111cee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:02.590703 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3ea91-bb3b-489a-a491-04192111cee3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:02.601693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc3ea91-bb3b-489a-a491-04192111cee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:02.603719 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc3ea91-bb3b-489a-a491-04192111cee3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:02.610956 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488ce72-0540-4aea-917b-00d0043d1856 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:02.620769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-cbc3ea91-bb3b-489a-a491-04192111cee3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:18:02.620769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:18:02.620769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:18:02.620769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:18:02.620769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:18:02.620769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:18:02.620769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:18:02.620769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:18:02.620769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:18:02.620769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:18:02.620769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:18:02.620769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:18:02.620769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:18:02.620769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:18:02.620769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:18:02.621835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:18:02.621835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:18:02.621835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:18:02.621835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:18:02.621835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:18:02.621835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:18:02.621835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:18:02.621835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:18:02.621835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:18:02.621835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:18:02.621835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:18:02.621835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:18:02.621835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:18:02.621835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:18:02.621835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:18:02.621835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:18:02.621835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:18:02.623189 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:18:02.623189 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:18:02.623189 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:18:02.623189 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:18:02.623189 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 543/1092] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:18:02.627585 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3ea91-bb3b-489a-a491-04192111cee3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:02.646854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89424c-2c29-447f-a3d2-4a09eff9c2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:02.647529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89424c-2c29-447f-a3d2-4a09eff9c2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:02.647820 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89424c-2c29-447f-a3d2-4a09eff9c2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:02.648100 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89424c-2c29-447f-a3d2-4a09eff9c2f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:02.649382 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b89424c-2c29-447f-a3d2-4a09eff9c2f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:02.649697 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b89424c-2c29-447f-a3d2-4a09eff9c2f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:02.651087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b89424c-2c29-447f-a3d2-4a09eff9c2f1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:02.659633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 550/1093] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:18:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:18:02.665953 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89424c-2c29-447f-a3d2-4a09eff9c2f1 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:02.734353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3ea91-bb3b-489a-a491-04192111cee3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:02.751656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:02.768815 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd40a8-f380-406a-984a-b2bfaaca7c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:02.769637 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd40a8-f380-406a-984a-b2bfaaca7c10 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:02.769925 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd40a8-f380-406a-984a-b2bfaaca7c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:02.770202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd40a8-f380-406a-984a-b2bfaaca7c10 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:02.770924 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd40a8-f380-406a-984a-b2bfaaca7c10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:02.771228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd40a8-f380-406a-984a-b2bfaaca7c10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:02.772218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89424c-2c29-447f-a3d2-4a09eff9c2f1 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:02.772598 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd40a8-f380-406a-984a-b2bfaaca7c10 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:02.790081 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 544/1094] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:18:02.791395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd40a8-f380-406a-984a-b2bfaaca7c10 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:02.807854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce167ea4-363d-448a-a59a-a043addaebab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:02.808733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce167ea4-363d-448a-a59a-a043addaebab None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:02.809024 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce167ea4-363d-448a-a59a-a043addaebab None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:02.809327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce167ea4-363d-448a-a59a-a043addaebab None admin] PATH_INFO: `/v3/users/b550ec0df73b44ff987929104bec3c36` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:02.810315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:02.810544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce167ea4-363d-448a-a59a-a043addaebab None admin] RBAC: Authorizing `identity:get_user(user_id=b550ec0df73b44ff987929104bec3c36)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:02.812099 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce167ea4-363d-448a-a59a-a043addaebab None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:02.813919 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 551/1095] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:18:02 2022] GET /identity/v3/users/b550ec0df73b44ff987929104bec3c36 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:18:02.819544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce167ea4-363d-448a-a59a-a043addaebab None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:02.894083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd40a8-f380-406a-984a-b2bfaaca7c10 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:02.924151 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce167ea4-363d-448a-a59a-a043addaebab None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:02.932276 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f44cf8-c7f2-4c89-97e2-0994825d335c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:02.933539 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f44cf8-c7f2-4c89-97e2-0994825d335c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:02.933827 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f44cf8-c7f2-4c89-97e2-0994825d335c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:02.934423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f44cf8-c7f2-4c89-97e2-0994825d335c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:02.942559 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f44cf8-c7f2-4c89-97e2-0994825d335c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:02.944090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f44cf8-c7f2-4c89-97e2-0994825d335c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:02.960049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-42f44cf8-c7f2-4c89-97e2-0994825d335c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 06:18:02.960049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:18:02.960049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:18:02.960049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:18:02.960049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:18:02.960049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:18:02.960049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:18:02.960049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:18:02.960049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:18:02.960049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:18:02.960049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:18:02.960049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:18:02.960049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:18:02.960049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:18:02.960049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:18:02.960049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:18:02.961079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:18:02.962405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:18:02.962405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:18:02.962405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:18:02.962405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:18:02.962405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:18:02.962405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:18:02.962405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:18:02.962405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:18:02.962405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 06:18:02.962405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:18:02.962952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362fd13-d156-4398-953f-77e76995d2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:02.962952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362fd13-d156-4398-953f-77e76995d2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:02.962952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362fd13-d156-4398-953f-77e76995d2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:02.963154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 545/1096] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 06:18:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:18:02.963239 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362fd13-d156-4398-953f-77e76995d2f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:02.967189 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f44cf8-c7f2-4c89-97e2-0994825d335c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:02.985118 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:03.041649 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362fd13-d156-4398-953f-77e76995d2f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:03.043082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362fd13-d156-4398-953f-77e76995d2f9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:03.050870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:03.075379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f44cf8-c7f2-4c89-97e2-0994825d335c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:03.109556 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff8dff-6c9d-4174-a02a-6beef0a76aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:03.110231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff8dff-6c9d-4174-a02a-6beef0a76aed None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:03.110579 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff8dff-6c9d-4174-a02a-6beef0a76aed None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:03.110858 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff8dff-6c9d-4174-a02a-6beef0a76aed None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:03.111690 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff8dff-6c9d-4174-a02a-6beef0a76aed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:03.111996 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff8dff-6c9d-4174-a02a-6beef0a76aed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:03.113451 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff8dff-6c9d-4174-a02a-6beef0a76aed None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:03.132874 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 546/1097] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 06:18:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:18:03.138587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff8dff-6c9d-4174-a02a-6beef0a76aed None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:03.151152 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 552/1098] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:18:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 06:18:03.157716 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362fd13-d156-4398-953f-77e76995d2f9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:03.206831 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:03.245382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff8dff-6c9d-4174-a02a-6beef0a76aed None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:03.260388 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362fd13-d156-4398-953f-77e76995d2f9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:03.281686 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91b645-6ca0-42fe-8b56-87539d489717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:03.282359 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91b645-6ca0-42fe-8b56-87539d489717 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:03.282645 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91b645-6ca0-42fe-8b56-87539d489717 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:03.282932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91b645-6ca0-42fe-8b56-87539d489717 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:03.285760 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:03.297833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6de7b-412d-4116-a09a-17e4513855c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:03.298614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6de7b-412d-4116-a09a-17e4513855c5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:03.298906 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6de7b-412d-4116-a09a-17e4513855c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:03.299189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6de7b-412d-4116-a09a-17e4513855c5 None admin] PATH_INFO: `/v3/projects/d968876d970b4fba86f096e789b85ab1` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:03.300423 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6de7b-412d-4116-a09a-17e4513855c5 None admin] RBAC: Authorizing `identity:get_project(project_id=d968876d970b4fba86f096e789b85ab1)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:03.301918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6de7b-412d-4116-a09a-17e4513855c5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:03.303624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 553/1099] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:18:03 2022] GET /identity/v3/projects/d968876d970b4fba86f096e789b85ab1 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:18:03.310540 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6de7b-412d-4116-a09a-17e4513855c5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:03.373822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91b645-6ca0-42fe-8b56-87539d489717 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:03.375275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91b645-6ca0-42fe-8b56-87539d489717 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:03.415440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6de7b-412d-4116-a09a-17e4513855c5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:03.432494 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:03.453100 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cf9c9-4a60-407b-abf8-0dd4bcc4b01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:03.453797 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cf9c9-4a60-407b-abf8-0dd4bcc4b01e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:03.454084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cf9c9-4a60-407b-abf8-0dd4bcc4b01e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:03.454363 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cf9c9-4a60-407b-abf8-0dd4bcc4b01e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:03.455194 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508cf9c9-4a60-407b-abf8-0dd4bcc4b01e None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=d968876d970b4fba86f096e789b85ab1, user.id=b550ec0df73b44ff987929104bec3c36) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:03.455511 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508cf9c9-4a60-407b-abf8-0dd4bcc4b01e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:03.457219 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508cf9c9-4a60-407b-abf8-0dd4bcc4b01e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:03.463048 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 554/1100] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:18:03 2022] GET /identity/v3/role_assignments?user.id=b550ec0df73b44ff987929104bec3c36&scope.project.id=d968876d970b4fba86f096e789b85ab1&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:18:03.485104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 547/1101] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:18:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 06:18:03.491118 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cf9c9-4a60-407b-abf8-0dd4bcc4b01e None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:03.620160 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cf9c9-4a60-407b-abf8-0dd4bcc4b01e None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:03.673947 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b941db2-7f5f-479c-aacd-4cad0ecda71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:03.674750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b941db2-7f5f-479c-aacd-4cad0ecda71d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:03.675033 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b941db2-7f5f-479c-aacd-4cad0ecda71d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:03.675308 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b941db2-7f5f-479c-aacd-4cad0ecda71d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:03.687646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b941db2-7f5f-479c-aacd-4cad0ecda71d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:03.689126 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b941db2-7f5f-479c-aacd-4cad0ecda71d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:03.708392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-3b941db2-7f5f-479c-aacd-4cad0ecda71d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:18:03.708392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:18:03.708392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:18:03.708392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:18:03.708392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:18:03.708392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:18:03.708392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:18:03.708392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:18:03.708392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:18:03.708392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:18:03.708392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:18:03.708392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:18:03.708392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:18:03.708392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:18:03.708392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:18:03.709619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:18:03.709619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:18:03.709619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:18:03.709619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:18:03.709619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:18:03.709619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:18:03.709619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:18:03.709619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:18:03.709619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:18:03.709619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:18:03.709619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:18:03.709619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:18:03.709619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:18:03.709619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:18:03.709619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:18:03.709619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:18:03.709619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:18:03.710898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:18:03.710898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:18:03.710898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:18:03.710898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:18:03.710898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 555/1102] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:18:03.715725 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91b645-6ca0-42fe-8b56-87539d489717 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:03.770246 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:03.884373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91b645-6ca0-42fe-8b56-87539d489717 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:03.921494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe26d9-a8d9-4327-86f3-7738e02db3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:03.922164 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe26d9-a8d9-4327-86f3-7738e02db3fe None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:03.922452 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe26d9-a8d9-4327-86f3-7738e02db3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:03.922738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe26d9-a8d9-4327-86f3-7738e02db3fe None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:03.923468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbe26d9-a8d9-4327-86f3-7738e02db3fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:03.923786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbe26d9-a8d9-4327-86f3-7738e02db3fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:03.925163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbe26d9-a8d9-4327-86f3-7738e02db3fe None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:03.937680 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 548/1103] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:03.943341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b941db2-7f5f-479c-aacd-4cad0ecda71d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:03.976502 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:04.052759 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b941db2-7f5f-479c-aacd-4cad0ecda71d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:04.091854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81e2a7-1ced-4095-a476-660a60d3e766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:04.092640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81e2a7-1ced-4095-a476-660a60d3e766 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:04.092923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81e2a7-1ced-4095-a476-660a60d3e766 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:04.093194 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81e2a7-1ced-4095-a476-660a60d3e766 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:04.100366 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81e2a7-1ced-4095-a476-660a60d3e766 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:04.101888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81e2a7-1ced-4095-a476-660a60d3e766 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:04.116350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-7c81e2a7-1ced-4095-a476-660a60d3e766 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:18:04.116350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:18:04.116350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:18:04.116350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:18:04.116350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:18:04.116350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:18:04.116350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:18:04.116350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:18:04.116350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:18:04.116350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:18:04.116350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:18:04.116350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:18:04.116350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:18:04.116350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:18:04.116350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:18:04.117988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:18:04.117988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:18:04.117988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:18:04.117988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:18:04.117988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:18:04.117988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:18:04.117988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:18:04.117988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:18:04.117988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:18:04.117988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:18:04.117988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:18:04.117988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:18:04.117988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:18:04.117988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:18:04.117988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:18:04.117988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:18:04.117988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:18:04.119662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:18:04.119662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:18:04.119662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:18:04.119662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:18:04.119662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:18:04.119662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:18:04.119662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:18:04.119662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:18:04.119662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 556/1104] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:18:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:18:04.126692 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe26d9-a8d9-4327-86f3-7738e02db3fe None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:04.233557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:04.240773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe26d9-a8d9-4327-86f3-7738e02db3fe None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:04.277821 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d2510-6799-49b6-9728-6f233d5341eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:04.278756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d2510-6799-49b6-9728-6f233d5341eb None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:04.279038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d2510-6799-49b6-9728-6f233d5341eb None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:04.279311 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d2510-6799-49b6-9728-6f233d5341eb None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:04.280108 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d2510-6799-49b6-9728-6f233d5341eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:04.280415 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d2510-6799-49b6-9728-6f233d5341eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:04.281898 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d2510-6799-49b6-9728-6f233d5341eb None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:04.295912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 549/1105] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:18:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:04.302025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81e2a7-1ced-4095-a476-660a60d3e766 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:04.401891 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:04.411504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81e2a7-1ced-4095-a476-660a60d3e766 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:04.447245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd999e-159f-436b-8f7a-bd4e8b78a255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:04.447912 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd999e-159f-436b-8f7a-bd4e8b78a255 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:04.448196 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd999e-159f-436b-8f7a-bd4e8b78a255 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:04.448473 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd999e-159f-436b-8f7a-bd4e8b78a255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:04.449331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd999e-159f-436b-8f7a-bd4e8b78a255 None admin] RBAC: Adding query filter params (role.id=1a4655a3aaf244afb3988418b88d766f, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=2a09f4878eac4d60a9a7dcedba15c05d) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:04.449645 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd999e-159f-436b-8f7a-bd4e8b78a255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:04.451107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd999e-159f-436b-8f7a-bd4e8b78a255 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:04.469321 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 557/1106] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:18:04 2022] GET /identity/v3/role_assignments?user.id=2a09f4878eac4d60a9a7dcedba15c05d&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=1a4655a3aaf244afb3988418b88d766f => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:18:05.838738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831973f-1476-4fd0-ad52-ca3fbd359437 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:05.839272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831973f-1476-4fd0-ad52-ca3fbd359437 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:05.839272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831973f-1476-4fd0-ad52-ca3fbd359437 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:05.840672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 550/1107] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:05.846106 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75aacc4-c217-43f3-9605-19e0d6f0f890 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:05.846531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75aacc4-c217-43f3-9605-19e0d6f0f890 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:05.846672 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75aacc4-c217-43f3-9605-19e0d6f0f890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:05.863684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a75aacc4-c217-43f3-9605-19e0d6f0f890 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:05.864881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a75aacc4-c217-43f3-9605-19e0d6f0f890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:05.932051 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 558/1108] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:18:05.956468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831973f-1476-4fd0-ad52-ca3fbd359437 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:06.057880 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831973f-1476-4fd0-ad52-ca3fbd359437 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:06.091628 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7a65a-c54d-43ea-8dd0-1a3629bfd117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:06.092338 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7a65a-c54d-43ea-8dd0-1a3629bfd117 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:06.092621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7a65a-c54d-43ea-8dd0-1a3629bfd117 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:06.092898 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7a65a-c54d-43ea-8dd0-1a3629bfd117 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:06.094099 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7a65a-c54d-43ea-8dd0-1a3629bfd117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:06.095536 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7a65a-c54d-43ea-8dd0-1a3629bfd117 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:06.097157 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 551/1109] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:18:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:06.103434 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75aacc4-c217-43f3-9605-19e0d6f0f890 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:06.186339 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:06.201375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75aacc4-c217-43f3-9605-19e0d6f0f890 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:06.235166 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cb4b2-2385-410d-af19-9c8f60f13fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:06.235842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cb4b2-2385-410d-af19-9c8f60f13fc4 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:06.236129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cb4b2-2385-410d-af19-9c8f60f13fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:06.236405 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cb4b2-2385-410d-af19-9c8f60f13fc4 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:06.237117 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cb4b2-2385-410d-af19-9c8f60f13fc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:06.238594 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cb4b2-2385-410d-af19-9c8f60f13fc4 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:06.260660 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 559/1110] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:18:06 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:18:08.634811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a26f3b-59e7-435e-9b48-34833e797235 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:08.635320 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a26f3b-59e7-435e-9b48-34833e797235 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:08.635414 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a26f3b-59e7-435e-9b48-34833e797235 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:08.636754 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 552/1111] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:08.642025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c74c7-46c5-45f6-89e1-21831cf0af96 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:08.642396 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c74c7-46c5-45f6-89e1-21831cf0af96 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:08.642598 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c74c7-46c5-45f6-89e1-21831cf0af96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:08.659879 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-ba0c74c7-46c5-45f6-89e1-21831cf0af96 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:08.661065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ba0c74c7-46c5-45f6-89e1-21831cf0af96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:08.729254 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 560/1112] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:18:08.752086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a26f3b-59e7-435e-9b48-34833e797235 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:08.778589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:08.858560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a26f3b-59e7-435e-9b48-34833e797235 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:08.893270 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ebb19-2f4a-48d7-b241-e451de769c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:08.894083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ebb19-2f4a-48d7-b241-e451de769c81 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:08.894368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ebb19-2f4a-48d7-b241-e451de769c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:08.894647 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ebb19-2f4a-48d7-b241-e451de769c81 None admin] PATH_INFO: `/v3/roles/6e5dce7d61b34704ad2cd32251fa4d34` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:08.896137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ebb19-2f4a-48d7-b241-e451de769c81 None admin] RBAC: Authorizing `identity:get_role(role_id=6e5dce7d61b34704ad2cd32251fa4d34)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:08.897547 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ebb19-2f4a-48d7-b241-e451de769c81 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:08.900070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 553/1113] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:18:08 2022] GET /identity/v3/roles/6e5dce7d61b34704ad2cd32251fa4d34 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:08.906190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c74c7-46c5-45f6-89e1-21831cf0af96 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:08.975049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:09.004493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c74c7-46c5-45f6-89e1-21831cf0af96 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:09.040375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d9230-3d3b-4992-9cca-7f61225ac2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:09.041254 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d9230-3d3b-4992-9cca-7f61225ac2ad None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:09.041571 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d9230-3d3b-4992-9cca-7f61225ac2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:09.041849 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d9230-3d3b-4992-9cca-7f61225ac2ad None admin] PATH_INFO: `/v3/users/76631e17f31d484b936e2a933e1ba398` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:09.055637 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d9230-3d3b-4992-9cca-7f61225ac2ad None admin] RBAC: Authorizing `identity:get_user(user_id=76631e17f31d484b936e2a933e1ba398)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:09.057109 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d9230-3d3b-4992-9cca-7f61225ac2ad None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:09.058834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 561/1114] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:18:08 2022] GET /identity/v3/users/76631e17f31d484b936e2a933e1ba398 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:09.064532 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ebb19-2f4a-48d7-b241-e451de769c81 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:09.154474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:09.165168 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ebb19-2f4a-48d7-b241-e451de769c81 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:09.202160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0c240-86f6-4c6e-b01d-d801aa18ff43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:09.202851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0c240-86f6-4c6e-b01d-d801aa18ff43 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:09.203137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0c240-86f6-4c6e-b01d-d801aa18ff43 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:09.203418 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0c240-86f6-4c6e-b01d-d801aa18ff43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:09.290984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0c240-86f6-4c6e-b01d-d801aa18ff43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:09.293308 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0c240-86f6-4c6e-b01d-d801aa18ff43 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:09.392153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:09.403648 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 554/1115] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:18:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:18:09.409879 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d9230-3d3b-4992-9cca-7f61225ac2ad None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:09.511059 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d9230-3d3b-4992-9cca-7f61225ac2ad None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:09.540301 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:09.546805 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a3045-04df-4ff4-9e02-f3e4e8f3c2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:09.547600 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a3045-04df-4ff4-9e02-f3e4e8f3c2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:09.547885 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a3045-04df-4ff4-9e02-f3e4e8f3c2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:09.548161 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a3045-04df-4ff4-9e02-f3e4e8f3c2c1 None admin] PATH_INFO: `/v3/projects/d968876d970b4fba86f096e789b85ab1` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:09.549391 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a3045-04df-4ff4-9e02-f3e4e8f3c2c1 None admin] RBAC: Authorizing `identity:get_project(project_id=d968876d970b4fba86f096e789b85ab1)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:09.550903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a3045-04df-4ff4-9e02-f3e4e8f3c2c1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:09.552602 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 562/1116] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:18:09 2022] GET /identity/v3/projects/d968876d970b4fba86f096e789b85ab1 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:09.559374 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0c240-86f6-4c6e-b01d-d801aa18ff43 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:09.667229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0c240-86f6-4c6e-b01d-d801aa18ff43 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:09.705378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35de19-c895-412a-ac06-246a455dd0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:09.706050 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35de19-c895-412a-ac06-246a455dd0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:09.706336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35de19-c895-412a-ac06-246a455dd0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:09.706607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35de19-c895-412a-ac06-246a455dd0e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:09.707456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35de19-c895-412a-ac06-246a455dd0e4 None admin] RBAC: Adding query filter params (role.id=6e5dce7d61b34704ad2cd32251fa4d34, scope.project.id=d968876d970b4fba86f096e789b85ab1, user.id=76631e17f31d484b936e2a933e1ba398) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:09.707762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35de19-c895-412a-ac06-246a455dd0e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:09.709234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35de19-c895-412a-ac06-246a455dd0e4 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:09.715966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 555/1117] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:18:09 2022] GET /identity/v3/role_assignments?user.id=76631e17f31d484b936e2a933e1ba398&scope.project.id=d968876d970b4fba86f096e789b85ab1&role.id=6e5dce7d61b34704ad2cd32251fa4d34 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:11.931544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c6546-30b8-4980-b9e9-5d1f59369ee5 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:11.932114 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c6546-30b8-4980-b9e9-5d1f59369ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:11.932114 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c6546-30b8-4980-b9e9-5d1f59369ee5 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:11.933486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 563/1118] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:11.938662 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bad03-3bcd-4c9c-a984-29c2cb51c222 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:11.939095 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bad03-3bcd-4c9c-a984-29c2cb51c222 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:11.939230 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bad03-3bcd-4c9c-a984-29c2cb51c222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:11.956643 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b25bad03-3bcd-4c9c-a984-29c2cb51c222 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:11.957867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b25bad03-3bcd-4c9c-a984-29c2cb51c222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:12.008591 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:12.027564 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 556/1119] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:18:12.050177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c6546-30b8-4980-b9e9-5d1f59369ee5 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:12.154097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c6546-30b8-4980-b9e9-5d1f59369ee5 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:12.188386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35664a47-ee94-48c2-9925-adb3ce8c2931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:12.189161 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35664a47-ee94-48c2-9925-adb3ce8c2931 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:12.189478 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35664a47-ee94-48c2-9925-adb3ce8c2931 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:12.189760 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35664a47-ee94-48c2-9925-adb3ce8c2931 None admin] PATH_INFO: `/v3/roles/6e5dce7d61b34704ad2cd32251fa4d34` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:12.191224 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35664a47-ee94-48c2-9925-adb3ce8c2931 None admin] RBAC: Authorizing `identity:get_role(role_id=6e5dce7d61b34704ad2cd32251fa4d34)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:12.192618 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35664a47-ee94-48c2-9925-adb3ce8c2931 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:12.194032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 564/1120] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:18:12 2022] GET /identity/v3/roles/6e5dce7d61b34704ad2cd32251fa4d34 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:12.200686 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25bad03-3bcd-4c9c-a984-29c2cb51c222 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:12.300302 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25bad03-3bcd-4c9c-a984-29c2cb51c222 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:12.334281 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaebfb4-098d-482f-bc6a-ef5dbcf4f60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:12.335160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaebfb4-098d-482f-bc6a-ef5dbcf4f60c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:12.335439 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaebfb4-098d-482f-bc6a-ef5dbcf4f60c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:12.335794 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaebfb4-098d-482f-bc6a-ef5dbcf4f60c None admin] PATH_INFO: `/v3/users/76631e17f31d484b936e2a933e1ba398` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:12.337193 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaebfb4-098d-482f-bc6a-ef5dbcf4f60c None admin] RBAC: Authorizing `identity:get_user(user_id=76631e17f31d484b936e2a933e1ba398)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:12.338627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaebfb4-098d-482f-bc6a-ef5dbcf4f60c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:12.340271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 557/1121] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:18:12 2022] GET /identity/v3/users/76631e17f31d484b936e2a933e1ba398 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:12.346171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35664a47-ee94-48c2-9925-adb3ce8c2931 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:12.448134 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:12.489133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35664a47-ee94-48c2-9925-adb3ce8c2931 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:12.527323 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81d696-3b02-4bf1-bb06-012b4f739f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:12.528466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81d696-3b02-4bf1-bb06-012b4f739f84 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:12.528752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81d696-3b02-4bf1-bb06-012b4f739f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:12.529032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81d696-3b02-4bf1-bb06-012b4f739f84 None admin] PATH_INFO: `/v3/projects/d968876d970b4fba86f096e789b85ab1` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:12.530361 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81d696-3b02-4bf1-bb06-012b4f739f84 None admin] RBAC: Authorizing `identity:get_project(project_id=d968876d970b4fba86f096e789b85ab1)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:12.531854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81d696-3b02-4bf1-bb06-012b4f739f84 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:12.533694 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 565/1122] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:18:12 2022] GET /identity/v3/projects/d968876d970b4fba86f096e789b85ab1 => generated 311 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:12.540329 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaebfb4-098d-482f-bc6a-ef5dbcf4f60c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:12.596061 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:12.646091 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaebfb4-098d-482f-bc6a-ef5dbcf4f60c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:12.682982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b6666-1053-4451-a90b-2d8b0d7092ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:12.683872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b6666-1053-4451-a90b-2d8b0d7092ed None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:12.684151 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b6666-1053-4451-a90b-2d8b0d7092ed None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:12.684423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b6666-1053-4451-a90b-2d8b0d7092ed None admin] PATH_INFO: `/v3/projects/d968876d970b4fba86f096e789b85ab1/users/76631e17f31d484b936e2a933e1ba398/roles/6e5dce7d61b34704ad2cd32251fa4d34` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:12.686626 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b6666-1053-4451-a90b-2d8b0d7092ed None admin] RBAC: Authorizing `identity:create_grant(project_id=d968876d970b4fba86f096e789b85ab1, user_id=76631e17f31d484b936e2a933e1ba398, role_id=6e5dce7d61b34704ad2cd32251fa4d34)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:12.688048 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b6666-1053-4451-a90b-2d8b0d7092ed None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:12.702969 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 558/1123] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:18:12 2022] PUT /identity/v3/projects/d968876d970b4fba86f096e789b85ab1/users/76631e17f31d484b936e2a933e1ba398/roles/6e5dce7d61b34704ad2cd32251fa4d34 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:18:14.909140 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714e193-cfb9-4664-86b5-fcb4f2614d2e None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:14.909748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714e193-cfb9-4664-86b5-fcb4f2614d2e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:14.909748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714e193-cfb9-4664-86b5-fcb4f2614d2e None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:14.911102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 566/1124] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:14.916429 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17d25c-4d9c-44b1-bbb9-ff4656a2abbc None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:14.916861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17d25c-4d9c-44b1-bbb9-ff4656a2abbc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:14.916974 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17d25c-4d9c-44b1-bbb9-ff4656a2abbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:14.933738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-bb17d25c-4d9c-44b1-bbb9-ff4656a2abbc None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:14.934912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bb17d25c-4d9c-44b1-bbb9-ff4656a2abbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:15.002445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 559/1125] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:18:15.027714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714e193-cfb9-4664-86b5-fcb4f2614d2e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:15.128308 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714e193-cfb9-4664-86b5-fcb4f2614d2e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:15.163164 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915c2a0-5a51-465c-bcb4-b28a2e8a572a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:15.163962 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915c2a0-5a51-465c-bcb4-b28a2e8a572a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:15.164258 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915c2a0-5a51-465c-bcb4-b28a2e8a572a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:15.164628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915c2a0-5a51-465c-bcb4-b28a2e8a572a None admin] PATH_INFO: `/v3/roles/6e5dce7d61b34704ad2cd32251fa4d34` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:15.166139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915c2a0-5a51-465c-bcb4-b28a2e8a572a None admin] RBAC: Authorizing `identity:get_role(role_id=6e5dce7d61b34704ad2cd32251fa4d34)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:15.167528 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915c2a0-5a51-465c-bcb4-b28a2e8a572a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:15.168914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 567/1126] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:18:15 2022] GET /identity/v3/roles/6e5dce7d61b34704ad2cd32251fa4d34 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:15.174768 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17d25c-4d9c-44b1-bbb9-ff4656a2abbc None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:15.219570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:15.282071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17d25c-4d9c-44b1-bbb9-ff4656a2abbc None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:15.318106 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f4291-4b30-4dc0-9461-44151afcc5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:15.318992 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f4291-4b30-4dc0-9461-44151afcc5db None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:15.319274 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f4291-4b30-4dc0-9461-44151afcc5db None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:15.319548 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f4291-4b30-4dc0-9461-44151afcc5db None admin] PATH_INFO: `/v3/users/76631e17f31d484b936e2a933e1ba398` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:15.321271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f4291-4b30-4dc0-9461-44151afcc5db None admin] RBAC: Authorizing `identity:get_user(user_id=76631e17f31d484b936e2a933e1ba398)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:15.322781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f4291-4b30-4dc0-9461-44151afcc5db None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:15.324530 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 560/1127] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:18:15 2022] GET /identity/v3/users/76631e17f31d484b936e2a933e1ba398 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:15.330448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915c2a0-5a51-465c-bcb4-b28a2e8a572a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:15.333796 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:15.434316 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915c2a0-5a51-465c-bcb4-b28a2e8a572a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:15.468643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c40f4b-f5c5-4306-a118-ee75a193500a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:15.469316 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c40f4b-f5c5-4306-a118-ee75a193500a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:15.469617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c40f4b-f5c5-4306-a118-ee75a193500a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:15.469899 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c40f4b-f5c5-4306-a118-ee75a193500a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:15.535539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c40f4b-f5c5-4306-a118-ee75a193500a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:15.537010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c40f4b-f5c5-4306-a118-ee75a193500a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:15.539345 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:15.634904 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 568/1128] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:18:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:18:15.640876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f4291-4b30-4dc0-9461-44151afcc5db None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:15.741682 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f4291-4b30-4dc0-9461-44151afcc5db None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:15.753987 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:15.777548 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41b5aa-46ce-41c8-84d5-66a559523828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:15.778659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41b5aa-46ce-41c8-84d5-66a559523828 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:15.778942 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41b5aa-46ce-41c8-84d5-66a559523828 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:15.779220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41b5aa-46ce-41c8-84d5-66a559523828 None admin] PATH_INFO: `/v3/projects/d968876d970b4fba86f096e789b85ab1` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:15.780405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41b5aa-46ce-41c8-84d5-66a559523828 None admin] RBAC: Authorizing `identity:get_project(project_id=d968876d970b4fba86f096e789b85ab1)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:15.781883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41b5aa-46ce-41c8-84d5-66a559523828 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:15.783586 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 561/1129] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:18:15 2022] GET /identity/v3/projects/d968876d970b4fba86f096e789b85ab1 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:15.790390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c40f4b-f5c5-4306-a118-ee75a193500a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:15.886255 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c40f4b-f5c5-4306-a118-ee75a193500a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:15.887757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:15.922394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ba3de-3bf2-4421-9093-02e5973ee3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:15.923052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ba3de-3bf2-4421-9093-02e5973ee3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:15.923315 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ba3de-3bf2-4421-9093-02e5973ee3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:15.923590 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ba3de-3bf2-4421-9093-02e5973ee3c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:15.924407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ba3de-3bf2-4421-9093-02e5973ee3c2 None admin] RBAC: Adding query filter params (role.id=6e5dce7d61b34704ad2cd32251fa4d34, scope.project.id=d968876d970b4fba86f096e789b85ab1, user.id=76631e17f31d484b936e2a933e1ba398) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:15.924716 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ba3de-3bf2-4421-9093-02e5973ee3c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:15.926180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ba3de-3bf2-4421-9093-02e5973ee3c2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:15.931931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 569/1130] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:18:15 2022] GET /identity/v3/role_assignments?user.id=76631e17f31d484b936e2a933e1ba398&scope.project.id=d968876d970b4fba86f096e789b85ab1&role.id=6e5dce7d61b34704ad2cd32251fa4d34 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:18.198480 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aeb7b0-949e-48db-b48c-5bcf0024c8b0 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:18.199019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aeb7b0-949e-48db-b48c-5bcf0024c8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:18.199219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aeb7b0-949e-48db-b48c-5bcf0024c8b0 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:18.200787 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 562/1131] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:18.206101 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3982510-88af-4e96-a287-bfea10461bff None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:18.206504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3982510-88af-4e96-a287-bfea10461bff None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:18.206655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3982510-88af-4e96-a287-bfea10461bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:18.223673 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f3982510-88af-4e96-a287-bfea10461bff None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:18.224848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f3982510-88af-4e96-a287-bfea10461bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:18.292707 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 570/1132] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:18:18.316515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aeb7b0-949e-48db-b48c-5bcf0024c8b0 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:18.419152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aeb7b0-949e-48db-b48c-5bcf0024c8b0 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:18.452064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74b3e5-119b-4d8a-9d5e-bc7c22899c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:18.452776 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74b3e5-119b-4d8a-9d5e-bc7c22899c99 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:18.453060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74b3e5-119b-4d8a-9d5e-bc7c22899c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:18.453364 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74b3e5-119b-4d8a-9d5e-bc7c22899c99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:18.454561 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74b3e5-119b-4d8a-9d5e-bc7c22899c99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:18.456003 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74b3e5-119b-4d8a-9d5e-bc7c22899c99 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:18.457671 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 563/1133] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:18:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:18.464236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3982510-88af-4e96-a287-bfea10461bff None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:18.556322 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:18.562627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3982510-88af-4e96-a287-bfea10461bff None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:18.597144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94b1dc-9dd7-462e-8786-e3c263b5146a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:18.597842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94b1dc-9dd7-462e-8786-e3c263b5146a None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:18.598129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94b1dc-9dd7-462e-8786-e3c263b5146a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:18.598405 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94b1dc-9dd7-462e-8786-e3c263b5146a None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:18.599127 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94b1dc-9dd7-462e-8786-e3c263b5146a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:18.600571 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94b1dc-9dd7-462e-8786-e3c263b5146a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:18.626370 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 571/1134] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 06:18:18 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:18:20.911840 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea488767-50d0-4e5d-93c3-45aaa52d2d03 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:20.912382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea488767-50d0-4e5d-93c3-45aaa52d2d03 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:20.912382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea488767-50d0-4e5d-93c3-45aaa52d2d03 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:20.914283 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 564/1135] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:20.920013 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb5b36-45c5-4908-8ef8-aaf72ced021a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:20.920432 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb5b36-45c5-4908-8ef8-aaf72ced021a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:20.920575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb5b36-45c5-4908-8ef8-aaf72ced021a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:20.937729 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a4bb5b36-45c5-4908-8ef8-aaf72ced021a None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:20.938948 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a4bb5b36-45c5-4908-8ef8-aaf72ced021a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:21.004536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 572/1136] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:18:21.029683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea488767-50d0-4e5d-93c3-45aaa52d2d03 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:21.080381 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:21.134302 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea488767-50d0-4e5d-93c3-45aaa52d2d03 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:21.169059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948bf16-281b-45ba-9ac5-587ef1ce5699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:21.169789 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948bf16-281b-45ba-9ac5-587ef1ce5699 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:21.170072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948bf16-281b-45ba-9ac5-587ef1ce5699 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:21.170349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948bf16-281b-45ba-9ac5-587ef1ce5699 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:21.171511 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948bf16-281b-45ba-9ac5-587ef1ce5699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:21.172943 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948bf16-281b-45ba-9ac5-587ef1ce5699 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:21.174580 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 565/1137] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:18:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:21.180307 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb5b36-45c5-4908-8ef8-aaf72ced021a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:21.257427 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:21.280920 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb5b36-45c5-4908-8ef8-aaf72ced021a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:21.315458 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc87fe-8578-4f0f-a327-147ea127b5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:21.316124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc87fe-8578-4f0f-a327-147ea127b5d6 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:21.316410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc87fe-8578-4f0f-a327-147ea127b5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:21.316686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc87fe-8578-4f0f-a327-147ea127b5d6 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:21.317433 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc87fe-8578-4f0f-a327-147ea127b5d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:21.318879 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc87fe-8578-4f0f-a327-147ea127b5d6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:21.345721 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 573/1138] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:18:21 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:18:23.630123 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6aff6-0426-4b5f-8a89-2f31e680914e None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:23.630649 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6aff6-0426-4b5f-8a89-2f31e680914e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:23.630649 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6aff6-0426-4b5f-8a89-2f31e680914e None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:23.631960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 566/1139] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:23.637516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5901cfa-14f9-4ece-a35e-07d69b89d197 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:23.637938 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5901cfa-14f9-4ece-a35e-07d69b89d197 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:23.638075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5901cfa-14f9-4ece-a35e-07d69b89d197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:23.655259 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b5901cfa-14f9-4ece-a35e-07d69b89d197 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:23.656430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b5901cfa-14f9-4ece-a35e-07d69b89d197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:23.725341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 574/1140] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:18:23.748769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6aff6-0426-4b5f-8a89-2f31e680914e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:23.851945 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6aff6-0426-4b5f-8a89-2f31e680914e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:23.865910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:23.886171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f035a93-0767-426b-b466-ee4fe6483ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:23.886964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f035a93-0767-426b-b466-ee4fe6483ffe None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:23.887249 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f035a93-0767-426b-b466-ee4fe6483ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:23.887530 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f035a93-0767-426b-b466-ee4fe6483ffe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:23.907166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f035a93-0767-426b-b466-ee4fe6483ffe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:18:23.914798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-8f035a93-0767-426b-b466-ee4fe6483ffe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:18:23.914798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:18:23.914798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:18:23.914798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:18:23.914798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:18:23.914798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:18:23.914798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:18:23.914798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:18:23.914798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:18:23.914798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:18:23.914798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:18:23.914798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:18:23.914798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:18:23.914798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:18:23.914798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:18:23.914798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:18:23.916070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:18:23.916070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:18:23.916070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:18:23.916070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:18:23.916070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:18:23.916070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:18:23.916070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:18:23.916070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:18:23.916070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:18:23.916070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:18:23.916070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:18:23.916070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:18:23.916070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:18:23.916070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:18:23.916070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:18:23.916070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:18:23.917419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:18:23.917419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:18:23.917419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:18:23.917419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:18:23.917419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:18:23.917419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:18:23.917419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:18:23.917419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:18:23.917419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:18:23.917419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:18:23.917419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:18:23.917419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:18:23.917419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 567/1141] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:18:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:18:23.922057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5901cfa-14f9-4ece-a35e-07d69b89d197 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:23.982443 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:24.022115 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5901cfa-14f9-4ece-a35e-07d69b89d197 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:24.061678 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37510eb-f3bc-4b66-8e17-fe6e4db1eb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:24.062348 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37510eb-f3bc-4b66-8e17-fe6e4db1eb27 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:24.062627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37510eb-f3bc-4b66-8e17-fe6e4db1eb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:24.062898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37510eb-f3bc-4b66-8e17-fe6e4db1eb27 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:24.063638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37510eb-f3bc-4b66-8e17-fe6e4db1eb27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:24.063940 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37510eb-f3bc-4b66-8e17-fe6e4db1eb27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:24.065340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37510eb-f3bc-4b66-8e17-fe6e4db1eb27 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:24.074429 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 575/1142] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:18:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:24.080231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f035a93-0767-426b-b466-ee4fe6483ffe None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:24.179539 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f035a93-0767-426b-b466-ee4fe6483ffe None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:24.213091 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db9c86-f2f9-4eea-a1d0-542477c7799c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:24.213991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db9c86-f2f9-4eea-a1d0-542477c7799c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:24.214278 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db9c86-f2f9-4eea-a1d0-542477c7799c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:24.214556 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db9c86-f2f9-4eea-a1d0-542477c7799c None admin] PATH_INFO: `/v3/users/eb5cba58af7c4b8abd1d8c16ecc9740c` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:24.229362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db9c86-f2f9-4eea-a1d0-542477c7799c None admin] RBAC: Authorizing `identity:get_user(user_id=eb5cba58af7c4b8abd1d8c16ecc9740c)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:24.230828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db9c86-f2f9-4eea-a1d0-542477c7799c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:24.232563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 568/1143] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:18:24 2022] GET /identity/v3/users/eb5cba58af7c4b8abd1d8c16ecc9740c => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:24.238317 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37510eb-f3bc-4b66-8e17-fe6e4db1eb27 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:24.339211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37510eb-f3bc-4b66-8e17-fe6e4db1eb27 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:24.364749 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:24.373859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b12357-c7b1-42d5-8038-aecf02bd0367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:24.374508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b12357-c7b1-42d5-8038-aecf02bd0367 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:24.374793 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b12357-c7b1-42d5-8038-aecf02bd0367 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:24.375067 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b12357-c7b1-42d5-8038-aecf02bd0367 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:24.440971 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b12357-c7b1-42d5-8038-aecf02bd0367 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:24.442377 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b12357-c7b1-42d5-8038-aecf02bd0367 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:24.542147 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 576/1144] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:18:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:18:24.548041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db9c86-f2f9-4eea-a1d0-542477c7799c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:24.578907 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:24.648824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db9c86-f2f9-4eea-a1d0-542477c7799c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:24.682861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf649d9e-b0b0-4f0f-9808-ab0a3f32b2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:24.683724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf649d9e-b0b0-4f0f-9808-ab0a3f32b2ba None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:24.684009 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf649d9e-b0b0-4f0f-9808-ab0a3f32b2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:24.684286 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf649d9e-b0b0-4f0f-9808-ab0a3f32b2ba None admin] PATH_INFO: `/v3/projects/003960e2810d4fd9a779c443ff88c924` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:24.685500 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf649d9e-b0b0-4f0f-9808-ab0a3f32b2ba None admin] RBAC: Authorizing `identity:get_project(project_id=003960e2810d4fd9a779c443ff88c924)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:24.686950 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf649d9e-b0b0-4f0f-9808-ab0a3f32b2ba None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:24.688664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 569/1145] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:18:24 2022] GET /identity/v3/projects/003960e2810d4fd9a779c443ff88c924 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:24.695554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b12357-c7b1-42d5-8038-aecf02bd0367 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:24.723143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:24.804554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b12357-c7b1-42d5-8038-aecf02bd0367 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:24.838435 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627add6-6af5-46d1-9a0a-1dd7645e96e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:24.839101 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627add6-6af5-46d1-9a0a-1dd7645e96e1 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:24.839385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627add6-6af5-46d1-9a0a-1dd7645e96e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:24.839688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627add6-6af5-46d1-9a0a-1dd7645e96e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:24.840603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627add6-6af5-46d1-9a0a-1dd7645e96e1 None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=003960e2810d4fd9a779c443ff88c924, user.id=eb5cba58af7c4b8abd1d8c16ecc9740c) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:24.840920 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627add6-6af5-46d1-9a0a-1dd7645e96e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:24.842517 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627add6-6af5-46d1-9a0a-1dd7645e96e1 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:24.848140 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 577/1146] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:18:24 2022] GET /identity/v3/role_assignments?user.id=eb5cba58af7c4b8abd1d8c16ecc9740c&scope.project.id=003960e2810d4fd9a779c443ff88c924&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:27.053468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52823a2e-2d3b-4d3d-a93d-577c06c2c947 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:27.054618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52823a2e-2d3b-4d3d-a93d-577c06c2c947 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:27.054618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52823a2e-2d3b-4d3d-a93d-577c06c2c947 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:27.056003 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 570/1147] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:27.061253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10ba46-b809-4d80-8d1f-e842ecc34c7d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:27.061739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10ba46-b809-4d80-8d1f-e842ecc34c7d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:27.061840 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10ba46-b809-4d80-8d1f-e842ecc34c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:27.079826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-da10ba46-b809-4d80-8d1f-e842ecc34c7d None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:27.081042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-da10ba46-b809-4d80-8d1f-e842ecc34c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:27.150557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 578/1148] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:18:27.173644 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52823a2e-2d3b-4d3d-a93d-577c06c2c947 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:27.273097 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:27.277259 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52823a2e-2d3b-4d3d-a93d-577c06c2c947 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:27.311954 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fe91c-d721-40b6-92a7-3503c39b93cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:27.312663 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fe91c-d721-40b6-92a7-3503c39b93cf None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:27.312946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fe91c-d721-40b6-92a7-3503c39b93cf None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:27.313221 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fe91c-d721-40b6-92a7-3503c39b93cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:27.323891 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fe91c-d721-40b6-92a7-3503c39b93cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:18:27.331441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-623fe91c-d721-40b6-92a7-3503c39b93cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:18:27.331441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:18:27.331441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:18:27.331441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:18:27.331441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:18:27.331441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:18:27.331441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:18:27.331441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:18:27.331441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:18:27.331441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:18:27.331441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:18:27.331441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:18:27.331441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:18:27.331441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:18:27.331441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:18:27.331441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:18:27.332406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:18:27.332406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:18:27.332406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:18:27.332406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:18:27.332406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:18:27.332406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:18:27.332406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:18:27.332406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:18:27.332406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:18:27.332406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:18:27.332406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:18:27.332406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:18:27.332406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:18:27.332406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:18:27.332406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:18:27.332406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:18:27.333264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:18:27.333264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:18:27.333264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:18:27.333264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:18:27.333264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:18:27.333264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:18:27.333264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:18:27.333264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:18:27.333264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:18:27.333264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:18:27.333264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:18:27.333264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:18:27.333264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 571/1149] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:18:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:18:27.338974 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10ba46-b809-4d80-8d1f-e842ecc34c7d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:27.359946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:27.438144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10ba46-b809-4d80-8d1f-e842ecc34c7d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:27.471650 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ba24d-724a-46ae-91ab-1b99f231347a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:27.472330 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ba24d-724a-46ae-91ab-1b99f231347a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:27.472614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ba24d-724a-46ae-91ab-1b99f231347a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:27.472892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ba24d-724a-46ae-91ab-1b99f231347a None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:27.473671 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ba24d-724a-46ae-91ab-1b99f231347a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:27.473981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ba24d-724a-46ae-91ab-1b99f231347a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:27.475352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ba24d-724a-46ae-91ab-1b99f231347a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:27.483863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 579/1150] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:18:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:27.489938 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fe91c-d721-40b6-92a7-3503c39b93cf None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:27.590557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fe91c-d721-40b6-92a7-3503c39b93cf None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:27.627539 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787eae3e-5791-48b8-ac86-82b1e83b2f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:27.628426 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787eae3e-5791-48b8-ac86-82b1e83b2f09 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:27.628708 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787eae3e-5791-48b8-ac86-82b1e83b2f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:27.628988 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787eae3e-5791-48b8-ac86-82b1e83b2f09 None admin] PATH_INFO: `/v3/users/eb5cba58af7c4b8abd1d8c16ecc9740c` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:27.630315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787eae3e-5791-48b8-ac86-82b1e83b2f09 None admin] RBAC: Authorizing `identity:get_user(user_id=eb5cba58af7c4b8abd1d8c16ecc9740c)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:27.631783 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787eae3e-5791-48b8-ac86-82b1e83b2f09 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:27.633696 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 572/1151] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:18:27 2022] GET /identity/v3/users/eb5cba58af7c4b8abd1d8c16ecc9740c => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:27.639495 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ba24d-724a-46ae-91ab-1b99f231347a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:27.723731 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:27.741494 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ba24d-724a-46ae-91ab-1b99f231347a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:27.778211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c9e43-a0e7-414b-8a42-60f617737869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:27.779091 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c9e43-a0e7-414b-8a42-60f617737869 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:27.779375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c9e43-a0e7-414b-8a42-60f617737869 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:27.779654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c9e43-a0e7-414b-8a42-60f617737869 None admin] PATH_INFO: `/v3/projects/003960e2810d4fd9a779c443ff88c924` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:27.780884 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c9e43-a0e7-414b-8a42-60f617737869 None admin] RBAC: Authorizing `identity:get_project(project_id=003960e2810d4fd9a779c443ff88c924)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:27.782388 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c9e43-a0e7-414b-8a42-60f617737869 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:27.784127 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 580/1152] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:18:27 2022] GET /identity/v3/projects/003960e2810d4fd9a779c443ff88c924 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:27.791834 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787eae3e-5791-48b8-ac86-82b1e83b2f09 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:27.820785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:27.892063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787eae3e-5791-48b8-ac86-82b1e83b2f09 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:27.934126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802d737-b09e-46a3-9596-3baf75f8acb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:27.935015 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802d737-b09e-46a3-9596-3baf75f8acb3 None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:27.935345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802d737-b09e-46a3-9596-3baf75f8acb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:27.935581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802d737-b09e-46a3-9596-3baf75f8acb3 None admin] PATH_INFO: `/v3/projects/003960e2810d4fd9a779c443ff88c924/users/eb5cba58af7c4b8abd1d8c16ecc9740c/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:27.937568 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802d737-b09e-46a3-9596-3baf75f8acb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=003960e2810d4fd9a779c443ff88c924, user_id=eb5cba58af7c4b8abd1d8c16ecc9740c, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:27.939024 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802d737-b09e-46a3-9596-3baf75f8acb3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:27.953535 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 573/1153] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:18:27 2022] PUT /identity/v3/projects/003960e2810d4fd9a779c443ff88c924/users/eb5cba58af7c4b8abd1d8c16ecc9740c/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:18:30.525961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61a3dc-cc4f-4845-8c5f-e99b9974c855 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:30.526583 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61a3dc-cc4f-4845-8c5f-e99b9974c855 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:30.526583 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61a3dc-cc4f-4845-8c5f-e99b9974c855 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:30.528261 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 581/1154] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:30.534878 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8eb00-afd3-4f22-8806-7347fa447516 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:30.535580 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8eb00-afd3-4f22-8806-7347fa447516 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:30.535580 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8eb00-afd3-4f22-8806-7347fa447516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:30.552982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-fdc8eb00-afd3-4f22-8806-7347fa447516 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:30.554258 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fdc8eb00-afd3-4f22-8806-7347fa447516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:30.624810 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 574/1155] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:18:30.649253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61a3dc-cc4f-4845-8c5f-e99b9974c855 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:30.762993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61a3dc-cc4f-4845-8c5f-e99b9974c855 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:30.801000 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374989e-16f7-48a2-ac6d-a9bb7ea563fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:30.801860 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374989e-16f7-48a2-ac6d-a9bb7ea563fa None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:30.802178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374989e-16f7-48a2-ac6d-a9bb7ea563fa None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:30.802436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374989e-16f7-48a2-ac6d-a9bb7ea563fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:30.817529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374989e-16f7-48a2-ac6d-a9bb7ea563fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:18:30.826210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-b374989e-16f7-48a2-ac6d-a9bb7ea563fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:18:30.826210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:18:30.826210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:18:30.826210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:18:30.826210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:18:30.826210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:18:30.826210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:18:30.826210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:18:30.826210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:18:30.826210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:18:30.826210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:18:30.826210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:18:30.826210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:18:30.826210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:18:30.826210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:18:30.826210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:18:30.827535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:18:30.827535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:18:30.827535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:18:30.827535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:18:30.827535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:18:30.827535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:18:30.827535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:18:30.827535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:18:30.827535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:18:30.827535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:18:30.827535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:18:30.827535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:18:30.827535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:18:30.827535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:18:30.827535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:18:30.827535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:18:30.829102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:18:30.829102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:18:30.829102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:18:30.829102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:18:30.829102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:18:30.829102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:18:30.829102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:18:30.829102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:18:30.829102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:18:30.829102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:18:30.829102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:18:30.829102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:18:30.829102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 582/1156] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:18:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:18:30.836028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8eb00-afd3-4f22-8806-7347fa447516 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:30.858169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:30.949802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8eb00-afd3-4f22-8806-7347fa447516 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:30.983815 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd82a0a-30ed-46b4-9c09-659426db893b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:30.984479 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd82a0a-30ed-46b4-9c09-659426db893b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:30.984762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd82a0a-30ed-46b4-9c09-659426db893b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:30.985038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd82a0a-30ed-46b4-9c09-659426db893b None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:30.985819 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd82a0a-30ed-46b4-9c09-659426db893b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:30.986126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd82a0a-30ed-46b4-9c09-659426db893b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:30.987496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd82a0a-30ed-46b4-9c09-659426db893b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:30.996623 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 575/1157] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:18:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:31.002495 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374989e-16f7-48a2-ac6d-a9bb7ea563fa None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:31.016499 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:31.119954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374989e-16f7-48a2-ac6d-a9bb7ea563fa None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:31.157267 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c30c65-afb0-4706-9275-51acb725deb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:31.158172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c30c65-afb0-4706-9275-51acb725deb6 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:31.158459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c30c65-afb0-4706-9275-51acb725deb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:31.158785 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c30c65-afb0-4706-9275-51acb725deb6 None admin] PATH_INFO: `/v3/users/eb5cba58af7c4b8abd1d8c16ecc9740c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:31.160061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c30c65-afb0-4706-9275-51acb725deb6 None admin] RBAC: Authorizing `identity:get_user(user_id=eb5cba58af7c4b8abd1d8c16ecc9740c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:31.161546 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c30c65-afb0-4706-9275-51acb725deb6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:31.163384 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 583/1158] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:18:31 2022] GET /identity/v3/users/eb5cba58af7c4b8abd1d8c16ecc9740c => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:31.169676 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd82a0a-30ed-46b4-9c09-659426db893b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:31.259005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:31.280423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd82a0a-30ed-46b4-9c09-659426db893b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:31.318465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897c4a2-06e0-4f75-b2fa-277ea8faf29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:31.319144 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897c4a2-06e0-4f75-b2fa-277ea8faf29b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:31.319446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897c4a2-06e0-4f75-b2fa-277ea8faf29b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:31.319720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897c4a2-06e0-4f75-b2fa-277ea8faf29b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:31.392319 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897c4a2-06e0-4f75-b2fa-277ea8faf29b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:31.393816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897c4a2-06e0-4f75-b2fa-277ea8faf29b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:31.479190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:31.501382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 576/1159] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:18:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:18:31.508723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c30c65-afb0-4706-9275-51acb725deb6 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:31.597119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:31.622113 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c30c65-afb0-4706-9275-51acb725deb6 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:31.658308 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b844d7-704d-4a43-b82d-36fa68db2b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:31.659180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b844d7-704d-4a43-b82d-36fa68db2b54 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:31.659463 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b844d7-704d-4a43-b82d-36fa68db2b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:31.659737 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b844d7-704d-4a43-b82d-36fa68db2b54 None admin] PATH_INFO: `/v3/projects/003960e2810d4fd9a779c443ff88c924` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:31.660956 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b844d7-704d-4a43-b82d-36fa68db2b54 None admin] RBAC: Authorizing `identity:get_project(project_id=003960e2810d4fd9a779c443ff88c924)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:31.662464 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b844d7-704d-4a43-b82d-36fa68db2b54 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:31.664741 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 584/1160] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:18:31 2022] GET /identity/v3/projects/003960e2810d4fd9a779c443ff88c924 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:31.671075 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897c4a2-06e0-4f75-b2fa-277ea8faf29b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:31.775332 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897c4a2-06e0-4f75-b2fa-277ea8faf29b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:31.814374 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a9087-0109-425b-adc7-6fb342a4dc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:31.815061 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a9087-0109-425b-adc7-6fb342a4dc4d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:31.815348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a9087-0109-425b-adc7-6fb342a4dc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:31.815629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a9087-0109-425b-adc7-6fb342a4dc4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:31.816492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a9087-0109-425b-adc7-6fb342a4dc4d None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=003960e2810d4fd9a779c443ff88c924, user.id=eb5cba58af7c4b8abd1d8c16ecc9740c) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:31.816808 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a9087-0109-425b-adc7-6fb342a4dc4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:31.819442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a9087-0109-425b-adc7-6fb342a4dc4d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:31.826841 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 577/1161] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:18:31 2022] GET /identity/v3/role_assignments?user.id=eb5cba58af7c4b8abd1d8c16ecc9740c&scope.project.id=003960e2810d4fd9a779c443ff88c924&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:34.146109 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b4423-dcc4-4da8-a405-6dd460549a8a None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:34.146743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b4423-dcc4-4da8-a405-6dd460549a8a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:34.146743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b4423-dcc4-4da8-a405-6dd460549a8a None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:34.148848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 585/1162] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:34.155197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c0157-decb-47c3-aa72-617e50495a98 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:34.155689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c0157-decb-47c3-aa72-617e50495a98 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:34.155856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c0157-decb-47c3-aa72-617e50495a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:34.174384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8b7c0157-decb-47c3-aa72-617e50495a98 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:34.175575 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8b7c0157-decb-47c3-aa72-617e50495a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:34.218794 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:34.249588 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 578/1163] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:18:34.272806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b4423-dcc4-4da8-a405-6dd460549a8a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:34.384527 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b4423-dcc4-4da8-a405-6dd460549a8a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:34.426865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b703a-abeb-438f-a1d8-4d7da48ad6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:34.427436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b703a-abeb-438f-a1d8-4d7da48ad6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:34.427727 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b703a-abeb-438f-a1d8-4d7da48ad6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:34.428011 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b703a-abeb-438f-a1d8-4d7da48ad6b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:34.448213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:34.510320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b703a-abeb-438f-a1d8-4d7da48ad6b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:34.511974 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b703a-abeb-438f-a1d8-4d7da48ad6b8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:34.625458 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 586/1164] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:18:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:18:34.633181 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c0157-decb-47c3-aa72-617e50495a98 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:34.743651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c0157-decb-47c3-aa72-617e50495a98 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:34.781769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5253083-0cb1-4d21-afbe-8074a25a6861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:34.782579 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5253083-0cb1-4d21-afbe-8074a25a6861 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:34.782868 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5253083-0cb1-4d21-afbe-8074a25a6861 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:34.783147 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5253083-0cb1-4d21-afbe-8074a25a6861 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:34.790637 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5253083-0cb1-4d21-afbe-8074a25a6861 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:34.792247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5253083-0cb1-4d21-afbe-8074a25a6861 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:34.808212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-c5253083-0cb1-4d21-afbe-8074a25a6861 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 06:18:34.808212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:18:34.808212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 06:18:34.808212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 06:18:34.808212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:18:34.808212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:18:34.808212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:18:34.808212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:18:34.808212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 06:18:34.808212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:18:34.808212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:18:34.808212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:18:34.808212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:18:34.808212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:18:34.808212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:18:34.808212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:18:34.808212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:18:34.809562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:18:34.809562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:18:34.809562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:18:34.809562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:18:34.809562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:18:34.809562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:18:34.809562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:18:34.809562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:18:34.809562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:18:34.809562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:18:34.809562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:18:34.809562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:18:34.809562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:18:34.809562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:18:34.809562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:18:34.809562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:18:34.810618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:18:34.810618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:18:34.810618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:18:34.810618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:18:34.810618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:18:34.810618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:18:34.810618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:18:34.810618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:18:34.810618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:18:34.810618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:18:34.810618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 06:18:34.810618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:18:34.810618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 06:18:34.810618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:18:34.810618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 579/1165] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 06:18:34 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:18:34.815954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b703a-abeb-438f-a1d8-4d7da48ad6b8 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:34.870352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:34.939211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b703a-abeb-438f-a1d8-4d7da48ad6b8 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:34.984091 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e260c5e-ef64-4734-b7d6-d254b6988693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:34.984761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e260c5e-ef64-4734-b7d6-d254b6988693 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:34.985049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e260c5e-ef64-4734-b7d6-d254b6988693 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:34.985373 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e260c5e-ef64-4734-b7d6-d254b6988693 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:34.986215 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e260c5e-ef64-4734-b7d6-d254b6988693 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:34.986526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e260c5e-ef64-4734-b7d6-d254b6988693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:34.988087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e260c5e-ef64-4734-b7d6-d254b6988693 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:34.997019 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 587/1166] 10.222.0.65 () {56 vars in 1115 bytes} [Thu May 19 06:18:34 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:37.290278 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035d985-0645-4c7e-bb00-16c827f2b0e0 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:37.290815 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035d985-0645-4c7e-bb00-16c827f2b0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:37.290815 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035d985-0645-4c7e-bb00-16c827f2b0e0 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:37.292216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 580/1167] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:37.297886 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d9258-1dab-4b50-a840-8b10a613f4e0 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:37.298388 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d9258-1dab-4b50-a840-8b10a613f4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:37.298502 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d9258-1dab-4b50-a840-8b10a613f4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:37.315483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-943d9258-1dab-4b50-a840-8b10a613f4e0 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:37.316685 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-943d9258-1dab-4b50-a840-8b10a613f4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:37.384715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 588/1168] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:18:37.407889 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035d985-0645-4c7e-bb00-16c827f2b0e0 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:37.443056 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:37.510663 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035d985-0645-4c7e-bb00-16c827f2b0e0 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:37.543823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbab3f9-e288-4ab1-aaa4-beef01fde590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:37.544394 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbab3f9-e288-4ab1-aaa4-beef01fde590 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:37.544678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbab3f9-e288-4ab1-aaa4-beef01fde590 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:37.544956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbab3f9-e288-4ab1-aaa4-beef01fde590 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:37.545728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbab3f9-e288-4ab1-aaa4-beef01fde590 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:37.547095 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbab3f9-e288-4ab1-aaa4-beef01fde590 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:37.563567 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 581/1169] 10.222.0.65 () {60 vars in 1138 bytes} [Thu May 19 06:18:37 2022] POST /identity/v3/domains => generated 267 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:18:39.880975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587992e-3cf3-4505-816c-d3b1c5948864 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:39.881758 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587992e-3cf3-4505-816c-d3b1c5948864 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:39.883440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587992e-3cf3-4505-816c-d3b1c5948864 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:39.884905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 589/1170] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:39.890905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3ccf4-8d31-4cc4-96e0-b46c943c4ed6 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:39.891381 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3ccf4-8d31-4cc4-96e0-b46c943c4ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:39.891470 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3ccf4-8d31-4cc4-96e0-b46c943c4ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:39.912586 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-6aa3ccf4-8d31-4cc4-96e0-b46c943c4ed6 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:39.914632 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6aa3ccf4-8d31-4cc4-96e0-b46c943c4ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:39.991699 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 582/1171] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:18:40.015065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587992e-3cf3-4505-816c-d3b1c5948864 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:40.060495 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:40.119218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587992e-3cf3-4505-816c-d3b1c5948864 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:40.153203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b879c9-45eb-43d4-9695-1775e76657ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:40.154029 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b879c9-45eb-43d4-9695-1775e76657ce None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:40.154317 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b879c9-45eb-43d4-9695-1775e76657ce None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:40.154595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b879c9-45eb-43d4-9695-1775e76657ce None admin] PATH_INFO: `/v3/domains/039cc9917a5e4d5fa1980e818885b4e3` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:40.165758 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b879c9-45eb-43d4-9695-1775e76657ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=039cc9917a5e4d5fa1980e818885b4e3)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:40.167220 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b879c9-45eb-43d4-9695-1775e76657ce None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:40.168906 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 590/1172] 10.222.0.65 () {56 vars in 1183 bytes} [Thu May 19 06:18:40 2022] GET /identity/v3/domains/039cc9917a5e4d5fa1980e818885b4e3 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:40.174762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3ccf4-8d31-4cc4-96e0-b46c943c4ed6 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:40.199155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:40.275146 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3ccf4-8d31-4cc4-96e0-b46c943c4ed6 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:40.308420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee7f21-912c-4d67-b0a4-479ee9f375ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:40.309079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee7f21-912c-4d67-b0a4-479ee9f375ed None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:40.309393 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee7f21-912c-4d67-b0a4-479ee9f375ed None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:40.309677 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee7f21-912c-4d67-b0a4-479ee9f375ed None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:40.310391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee7f21-912c-4d67-b0a4-479ee9f375ed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:40.311821 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee7f21-912c-4d67-b0a4-479ee9f375ed None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:40.339011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 583/1173] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 06:18:40 2022] POST /identity/v3/projects => generated 361 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:18:42.644682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0702f1f-6a2e-4e75-89d3-a348ec72c70e None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:42.645203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0702f1f-6a2e-4e75-89d3-a348ec72c70e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:42.645323 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0702f1f-6a2e-4e75-89d3-a348ec72c70e None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:42.646697 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 591/1174] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:42.651949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec57d9-5a1a-46b6-955e-8ca14b1a0b45 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:42.652398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec57d9-5a1a-46b6-955e-8ca14b1a0b45 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:42.652494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec57d9-5a1a-46b6-955e-8ca14b1a0b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:42.669936 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c3ec57d9-5a1a-46b6-955e-8ca14b1a0b45 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:42.671118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c3ec57d9-5a1a-46b6-955e-8ca14b1a0b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:42.739972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 584/1175] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:18:42.766506 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0702f1f-6a2e-4e75-89d3-a348ec72c70e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:42.863325 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:42.867706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0702f1f-6a2e-4e75-89d3-a348ec72c70e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:42.901422 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585f068-43ef-4627-83a9-640efd3d9b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:42.902203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585f068-43ef-4627-83a9-640efd3d9b91 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:42.902496 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585f068-43ef-4627-83a9-640efd3d9b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:42.903027 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585f068-43ef-4627-83a9-640efd3d9b91 None admin] PATH_INFO: `/v3/domains/039cc9917a5e4d5fa1980e818885b4e3` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:42.904204 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4585f068-43ef-4627-83a9-640efd3d9b91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=039cc9917a5e4d5fa1980e818885b4e3)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:42.905685 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4585f068-43ef-4627-83a9-640efd3d9b91 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:42.907313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 592/1176] 10.222.0.65 () {56 vars in 1183 bytes} [Thu May 19 06:18:42 2022] GET /identity/v3/domains/039cc9917a5e4d5fa1980e818885b4e3 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:42.913168 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec57d9-5a1a-46b6-955e-8ca14b1a0b45 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:42.921372 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:43.014274 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec57d9-5a1a-46b6-955e-8ca14b1a0b45 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:43.052882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc17777-402c-4923-9b30-97fee694a64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:43.054204 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc17777-402c-4923-9b30-97fee694a64e None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:43.054488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc17777-402c-4923-9b30-97fee694a64e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:43.054765 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc17777-402c-4923-9b30-97fee694a64e None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:43.055507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc17777-402c-4923-9b30-97fee694a64e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:43.056953 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc17777-402c-4923-9b30-97fee694a64e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:43.078555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 585/1177] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:18:42 2022] POST /identity/v3/users => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:18:45.357251 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041eab9-f017-4a48-961f-22ec0a2c986c None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:45.357771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041eab9-f017-4a48-961f-22ec0a2c986c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:45.357859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041eab9-f017-4a48-961f-22ec0a2c986c None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:45.359210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 593/1178] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:45.364710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbb4a5-77b0-447a-b180-0f1ebba177ea None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:45.365119 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbb4a5-77b0-447a-b180-0f1ebba177ea None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:45.365271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbb4a5-77b0-447a-b180-0f1ebba177ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:45.382117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3abbb4a5-77b0-447a-b180-0f1ebba177ea None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:45.383313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3abbb4a5-77b0-447a-b180-0f1ebba177ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:45.439796 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:45.451191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 586/1179] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:18:45.475951 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041eab9-f017-4a48-961f-22ec0a2c986c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:45.576992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041eab9-f017-4a48-961f-22ec0a2c986c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:45.609935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360468c0-e0d6-431c-aa57-8d1fd79d4add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:45.610649 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360468c0-e0d6-431c-aa57-8d1fd79d4add None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:45.610932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360468c0-e0d6-431c-aa57-8d1fd79d4add None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:45.611208 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360468c0-e0d6-431c-aa57-8d1fd79d4add None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:45.621119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360468c0-e0d6-431c-aa57-8d1fd79d4add None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:18:45.628633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-360468c0-e0d6-431c-aa57-8d1fd79d4add None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:18:45.628633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:18:45.628633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:18:45.628633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:18:45.628633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:18:45.628633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:18:45.628633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:18:45.628633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:18:45.628633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:18:45.628633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:18:45.628633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:18:45.628633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:18:45.628633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:18:45.628633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:18:45.628633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:18:45.628633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:18:45.629800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:18:45.629800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:18:45.629800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:18:45.629800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:18:45.629800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:18:45.629800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:18:45.629800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:18:45.629800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:18:45.629800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:18:45.629800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:18:45.629800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:18:45.629800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:18:45.629800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:18:45.629800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:18:45.629800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:18:45.629800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:18:45.630848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:18:45.630848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:18:45.630848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:18:45.630848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:18:45.630848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:18:45.630848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:18:45.630848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:18:45.630848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:18:45.630848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:18:45.630848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:18:45.630848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:18:45.630848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:18:45.630848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 594/1180] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:18:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:18:45.636418 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbb4a5-77b0-447a-b180-0f1ebba177ea None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:45.734168 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbb4a5-77b0-447a-b180-0f1ebba177ea None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:45.767454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5dc63c-a828-4bb8-81c2-f01a5174e6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:45.768131 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5dc63c-a828-4bb8-81c2-f01a5174e6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:45.768415 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5dc63c-a828-4bb8-81c2-f01a5174e6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:45.768694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5dc63c-a828-4bb8-81c2-f01a5174e6b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:45.769474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5dc63c-a828-4bb8-81c2-f01a5174e6b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:45.769785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5dc63c-a828-4bb8-81c2-f01a5174e6b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:45.771146 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5dc63c-a828-4bb8-81c2-f01a5174e6b8 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:45.780605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 587/1181] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:18:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:45.786585 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360468c0-e0d6-431c-aa57-8d1fd79d4add None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:45.816333 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:45.889609 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360468c0-e0d6-431c-aa57-8d1fd79d4add None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:45.922288 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf3544-8c80-45e8-a001-1e865e29a244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:45.923147 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf3544-8c80-45e8-a001-1e865e29a244 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:45.923437 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf3544-8c80-45e8-a001-1e865e29a244 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:45.923713 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf3544-8c80-45e8-a001-1e865e29a244 None admin] PATH_INFO: `/v3/users/d22bce40b6f44fa8b659ed5870add8e2` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:45.937699 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accf3544-8c80-45e8-a001-1e865e29a244 None admin] RBAC: Authorizing `identity:get_user(user_id=d22bce40b6f44fa8b659ed5870add8e2)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:45.939174 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accf3544-8c80-45e8-a001-1e865e29a244 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:45.940878 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 595/1182] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:18:45 2022] GET /identity/v3/users/d22bce40b6f44fa8b659ed5870add8e2 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:45.948038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5dc63c-a828-4bb8-81c2-f01a5174e6b8 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:46.003037 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:46.051773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5dc63c-a828-4bb8-81c2-f01a5174e6b8 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:46.089293 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0c1b8-a0fc-4403-adf8-993f79273e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:46.089977 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0c1b8-a0fc-4403-adf8-993f79273e8c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:46.090264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0c1b8-a0fc-4403-adf8-993f79273e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:46.090541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0c1b8-a0fc-4403-adf8-993f79273e8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:46.161165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0c1b8-a0fc-4403-adf8-993f79273e8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:46.163161 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0c1b8-a0fc-4403-adf8-993f79273e8c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:46.218138 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:46.265075 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 588/1183] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:18:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:18:46.271032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf3544-8c80-45e8-a001-1e865e29a244 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:46.359214 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:46.374456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf3544-8c80-45e8-a001-1e865e29a244 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:46.407388 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf4bdb-d891-4a3d-8309-4232e6a75d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:46.408244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf4bdb-d891-4a3d-8309-4232e6a75d4e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:46.408529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf4bdb-d891-4a3d-8309-4232e6a75d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:46.408806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf4bdb-d891-4a3d-8309-4232e6a75d4e None admin] PATH_INFO: `/v3/projects/b1fcfccf1d4f4dec809d9d4d30680974` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:46.409959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf4bdb-d891-4a3d-8309-4232e6a75d4e None admin] RBAC: Authorizing `identity:get_project(project_id=b1fcfccf1d4f4dec809d9d4d30680974)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:46.411455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf4bdb-d891-4a3d-8309-4232e6a75d4e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:46.413848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 596/1184] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:18:46 2022] GET /identity/v3/projects/b1fcfccf1d4f4dec809d9d4d30680974 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:46.420171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0c1b8-a0fc-4403-adf8-993f79273e8c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:46.517836 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0c1b8-a0fc-4403-adf8-993f79273e8c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:46.552148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce2655-afaf-48da-8e11-320ec1d167d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:46.552859 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce2655-afaf-48da-8e11-320ec1d167d1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:46.553853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce2655-afaf-48da-8e11-320ec1d167d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:46.554140 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce2655-afaf-48da-8e11-320ec1d167d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:46.554976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce2655-afaf-48da-8e11-320ec1d167d1 None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=b1fcfccf1d4f4dec809d9d4d30680974, user.id=d22bce40b6f44fa8b659ed5870add8e2) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:46.555289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce2655-afaf-48da-8e11-320ec1d167d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:46.556730 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce2655-afaf-48da-8e11-320ec1d167d1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:46.562241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 589/1185] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:18:46 2022] GET /identity/v3/role_assignments?user.id=d22bce40b6f44fa8b659ed5870add8e2&scope.project.id=b1fcfccf1d4f4dec809d9d4d30680974&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:48.786375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abf2e6-9401-4f81-8ca6-b9d12f635fdd None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:48.786933 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abf2e6-9401-4f81-8ca6-b9d12f635fdd None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:48.786933 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abf2e6-9401-4f81-8ca6-b9d12f635fdd None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:48.788378 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 597/1186] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:48.793672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a892-888d-4f53-b56a-f07287d1f390 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:48.794075 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a892-888d-4f53-b56a-f07287d1f390 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:48.794245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a892-888d-4f53-b56a-f07287d1f390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:48.810740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-1668a892-888d-4f53-b56a-f07287d1f390 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:48.811921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1668a892-888d-4f53-b56a-f07287d1f390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:48.825696 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:48.879854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 590/1187] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:18:48.903589 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abf2e6-9401-4f81-8ca6-b9d12f635fdd None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:49.004910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abf2e6-9401-4f81-8ca6-b9d12f635fdd None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:49.041113 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a3860-920d-4d50-96eb-22a029faade3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:49.041849 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a3860-920d-4d50-96eb-22a029faade3 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:49.042136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a3860-920d-4d50-96eb-22a029faade3 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:49.042414 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a3860-920d-4d50-96eb-22a029faade3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:49.052455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a3860-920d-4d50-96eb-22a029faade3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:18:49.060549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-197a3860-920d-4d50-96eb-22a029faade3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:18:49.060549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:18:49.060549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:18:49.060549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:18:49.060549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:18:49.060549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:18:49.060549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:18:49.060549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:18:49.060549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:18:49.060549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:18:49.060549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:18:49.060549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:18:49.060549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:18:49.060549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:18:49.060549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:18:49.060549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:18:49.061670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:18:49.061670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:18:49.061670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:18:49.061670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:18:49.061670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:18:49.061670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:18:49.061670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:18:49.061670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:18:49.061670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:18:49.061670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:18:49.061670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:18:49.061670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:18:49.061670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:18:49.061670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:18:49.061670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:18:49.061670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:18:49.062747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:18:49.062747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:18:49.062747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:18:49.062747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:18:49.062747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:18:49.062747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:18:49.062747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:18:49.062747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:18:49.062747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:18:49.062747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:18:49.062747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:18:49.062747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:18:49.062747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 598/1188] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:18:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:18:49.067950 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668a892-888d-4f53-b56a-f07287d1f390 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:49.165002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668a892-888d-4f53-b56a-f07287d1f390 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:49.197880 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fc929-2d62-4a29-9435-4656ec91c249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:49.198541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fc929-2d62-4a29-9435-4656ec91c249 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:49.198825 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fc929-2d62-4a29-9435-4656ec91c249 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:49.199101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fc929-2d62-4a29-9435-4656ec91c249 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:49.199850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119fc929-2d62-4a29-9435-4656ec91c249 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:49.200157 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119fc929-2d62-4a29-9435-4656ec91c249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:49.201552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119fc929-2d62-4a29-9435-4656ec91c249 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:49.209960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 591/1189] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:18:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:49.215788 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a3860-920d-4d50-96eb-22a029faade3 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:49.232045 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:49.319596 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a3860-920d-4d50-96eb-22a029faade3 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:49.355385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f46369-0772-4b25-910a-21fb1532afb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:49.356254 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f46369-0772-4b25-910a-21fb1532afb9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:49.356538 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f46369-0772-4b25-910a-21fb1532afb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:49.356813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f46369-0772-4b25-910a-21fb1532afb9 None admin] PATH_INFO: `/v3/users/d22bce40b6f44fa8b659ed5870add8e2` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:49.358090 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f46369-0772-4b25-910a-21fb1532afb9 None admin] RBAC: Authorizing `identity:get_user(user_id=d22bce40b6f44fa8b659ed5870add8e2)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:49.359552 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f46369-0772-4b25-910a-21fb1532afb9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:49.361170 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 599/1190] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:18:49 2022] GET /identity/v3/users/d22bce40b6f44fa8b659ed5870add8e2 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:49.367533 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fc929-2d62-4a29-9435-4656ec91c249 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:49.380134 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:49.465466 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fc929-2d62-4a29-9435-4656ec91c249 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:49.498298 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004eeeb-1370-46be-af57-477c4d34a338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:49.499157 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004eeeb-1370-46be-af57-477c4d34a338 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:49.499443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004eeeb-1370-46be-af57-477c4d34a338 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:49.499723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004eeeb-1370-46be-af57-477c4d34a338 None admin] PATH_INFO: `/v3/projects/b1fcfccf1d4f4dec809d9d4d30680974` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:49.500890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004eeeb-1370-46be-af57-477c4d34a338 None admin] RBAC: Authorizing `identity:get_project(project_id=b1fcfccf1d4f4dec809d9d4d30680974)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:49.502350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004eeeb-1370-46be-af57-477c4d34a338 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:49.504046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 592/1191] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:18:49 2022] GET /identity/v3/projects/b1fcfccf1d4f4dec809d9d4d30680974 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:49.510126 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f46369-0772-4b25-910a-21fb1532afb9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:49.595089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:49.614052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f46369-0772-4b25-910a-21fb1532afb9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:49.648248 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b11c3-8b8a-43cd-9701-986f98172e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:49.649137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b11c3-8b8a-43cd-9701-986f98172e4b None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:49.649455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b11c3-8b8a-43cd-9701-986f98172e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:49.649736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b11c3-8b8a-43cd-9701-986f98172e4b None admin] PATH_INFO: `/v3/projects/b1fcfccf1d4f4dec809d9d4d30680974/users/d22bce40b6f44fa8b659ed5870add8e2/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:49.651708 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b11c3-8b8a-43cd-9701-986f98172e4b None admin] RBAC: Authorizing `identity:create_grant(project_id=b1fcfccf1d4f4dec809d9d4d30680974, user_id=d22bce40b6f44fa8b659ed5870add8e2, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:49.653440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b11c3-8b8a-43cd-9701-986f98172e4b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:49.667043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 600/1192] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:18:49 2022] PUT /identity/v3/projects/b1fcfccf1d4f4dec809d9d4d30680974/users/d22bce40b6f44fa8b659ed5870add8e2/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:18:51.882503 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195faf4-7f9a-4e1e-9e07-f1802bb70057 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:51.883039 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195faf4-7f9a-4e1e-9e07-f1802bb70057 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:51.883039 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195faf4-7f9a-4e1e-9e07-f1802bb70057 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:51.884429 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 593/1193] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:18:51.890043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29a71b-c7d9-4ab4-8088-9b865a1fc684 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:51.890443 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29a71b-c7d9-4ab4-8088-9b865a1fc684 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:51.890604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29a71b-c7d9-4ab4-8088-9b865a1fc684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:51.907837 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0a29a71b-c7d9-4ab4-8088-9b865a1fc684 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:18:51.909031 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0a29a71b-c7d9-4ab4-8088-9b865a1fc684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:18:51.975282 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 601/1194] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:18:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:18:51.999907 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195faf4-7f9a-4e1e-9e07-f1802bb70057 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:52.085403 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:52.108895 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195faf4-7f9a-4e1e-9e07-f1802bb70057 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:52.142496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c70de-41fb-49e1-9c10-eb13adc6c3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:52.143210 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c70de-41fb-49e1-9c10-eb13adc6c3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:52.143494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c70de-41fb-49e1-9c10-eb13adc6c3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:52.143818 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c70de-41fb-49e1-9c10-eb13adc6c3a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:52.155232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c70de-41fb-49e1-9c10-eb13adc6c3a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:18:52.163353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-c50c70de-41fb-49e1-9c10-eb13adc6c3a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:18:52.163353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:18:52.163353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:18:52.163353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:18:52.163353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:18:52.163353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:18:52.163353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:18:52.163353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:18:52.163353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:18:52.163353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:18:52.163353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:18:52.163353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:18:52.163353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:18:52.163353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:18:52.163353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:18:52.163353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:18:52.164501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:18:52.164501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:18:52.164501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:18:52.164501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:18:52.164501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:18:52.164501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:18:52.164501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:18:52.164501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:18:52.164501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:18:52.164501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:18:52.164501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:18:52.164501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:18:52.164501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:18:52.164501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:18:52.164501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:18:52.164501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:18:52.165742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:18:52.165742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:18:52.165742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:18:52.165742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:18:52.165742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:18:52.165742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:18:52.165742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:18:52.165742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:18:52.165742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:18:52.165742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:18:52.165742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:18:52.165742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:18:52.165742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 594/1195] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:18:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:18:52.170626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29a71b-c7d9-4ab4-8088-9b865a1fc684 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:52.238015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:52.267847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29a71b-c7d9-4ab4-8088-9b865a1fc684 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:52.301760 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b64ac7-0683-4797-8bf5-27ca5f35c6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:52.302433 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b64ac7-0683-4797-8bf5-27ca5f35c6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:52.302857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b64ac7-0683-4797-8bf5-27ca5f35c6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:52.303170 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b64ac7-0683-4797-8bf5-27ca5f35c6e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:52.304034 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b64ac7-0683-4797-8bf5-27ca5f35c6e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:52.304341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b64ac7-0683-4797-8bf5-27ca5f35c6e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:52.305783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b64ac7-0683-4797-8bf5-27ca5f35c6e2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:52.315052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 602/1196] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:18:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:52.321649 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c70de-41fb-49e1-9c10-eb13adc6c3a9 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:52.420880 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c70de-41fb-49e1-9c10-eb13adc6c3a9 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:52.455112 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9b32e-157f-407c-86ab-d65159d57289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:52.455976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9b32e-157f-407c-86ab-d65159d57289 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:52.456258 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9b32e-157f-407c-86ab-d65159d57289 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:52.456534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9b32e-157f-407c-86ab-d65159d57289 None admin] PATH_INFO: `/v3/users/d22bce40b6f44fa8b659ed5870add8e2` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:52.457748 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9b32e-157f-407c-86ab-d65159d57289 None admin] RBAC: Authorizing `identity:get_user(user_id=d22bce40b6f44fa8b659ed5870add8e2)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:52.459758 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9b32e-157f-407c-86ab-d65159d57289 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:52.461453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 595/1197] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 06:18:52 2022] GET /identity/v3/users/d22bce40b6f44fa8b659ed5870add8e2 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:52.467208 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b64ac7-0683-4797-8bf5-27ca5f35c6e2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:52.567511 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b64ac7-0683-4797-8bf5-27ca5f35c6e2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:52.602358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b7737-799b-4953-b462-da833daceb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:52.603001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b7737-799b-4953-b462-da833daceb75 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:52.603291 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b7737-799b-4953-b462-da833daceb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:52.603572 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b7737-799b-4953-b462-da833daceb75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:52.606200 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:52.673798 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b7737-799b-4953-b462-da833daceb75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:52.675186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b7737-799b-4953-b462-da833daceb75 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:52.772901 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 603/1198] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:18:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:18:52.778987 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9b32e-157f-407c-86ab-d65159d57289 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:52.788459 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:52.876540 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9b32e-157f-407c-86ab-d65159d57289 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:52.910074 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6349a5-b379-43c1-a1cc-90ca1efd04be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:52.910930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6349a5-b379-43c1-a1cc-90ca1efd04be None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:52.911215 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6349a5-b379-43c1-a1cc-90ca1efd04be None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:52.911494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6349a5-b379-43c1-a1cc-90ca1efd04be None admin] PATH_INFO: `/v3/projects/b1fcfccf1d4f4dec809d9d4d30680974` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:52.913338 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6349a5-b379-43c1-a1cc-90ca1efd04be None admin] RBAC: Authorizing `identity:get_project(project_id=b1fcfccf1d4f4dec809d9d4d30680974)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:52.914764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6349a5-b379-43c1-a1cc-90ca1efd04be None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:52.916462 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 596/1199] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 06:18:52 2022] GET /identity/v3/projects/b1fcfccf1d4f4dec809d9d4d30680974 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:18:52.922521 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b7737-799b-4953-b462-da833daceb75 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:18:52.956182 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:18:53.025153 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b7737-799b-4953-b462-da833daceb75 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:18:53.057921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff17a0a-bd89-4c31-91f9-63af3275320d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:18:53.058576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff17a0a-bd89-4c31-91f9-63af3275320d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:18:53.058862 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff17a0a-bd89-4c31-91f9-63af3275320d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:18:53.059140 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff17a0a-bd89-4c31-91f9-63af3275320d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:18:53.059962 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff17a0a-bd89-4c31-91f9-63af3275320d None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=b1fcfccf1d4f4dec809d9d4d30680974, user.id=d22bce40b6f44fa8b659ed5870add8e2) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:18:53.060278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff17a0a-bd89-4c31-91f9-63af3275320d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:18:53.061757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff17a0a-bd89-4c31-91f9-63af3275320d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:18:53.067531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 604/1200] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:18:52 2022] GET /identity/v3/role_assignments?user.id=d22bce40b6f44fa8b659ed5870add8e2&scope.project.id=b1fcfccf1d4f4dec809d9d4d30680974&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:19:02.905953 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=63 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:19:03.067417 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=51 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:19:56.563113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c9aaf-b9bc-441f-8f74-61a1615920a3 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:19:56.563744 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c9aaf-b9bc-441f-8f74-61a1615920a3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:19:56.563744 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c9aaf-b9bc-441f-8f74-61a1615920a3 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:19:56.565718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 597/1201] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:19:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:19:56.571658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9067a8-5887-4597-9d2f-b16f363419dd None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:19:56.572350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9067a8-5887-4597-9d2f-b16f363419dd None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:19:56.572350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9067a8-5887-4597-9d2f-b16f363419dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:19:56.594539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9e9067a8-5887-4597-9d2f-b16f363419dd None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:19:56.595677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9e9067a8-5887-4597-9d2f-b16f363419dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:19:56.696341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 605/1202] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:19:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:19:56.722911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c9aaf-b9bc-441f-8f74-61a1615920a3 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:19:56.891041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c9aaf-b9bc-441f-8f74-61a1615920a3 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:19:56.967687 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2baaa-8561-4f45-ad97-b3a342889271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:19:56.970841 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2baaa-8561-4f45-ad97-b3a342889271 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:19:56.972408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2baaa-8561-4f45-ad97-b3a342889271 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:19:56.973133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2baaa-8561-4f45-ad97-b3a342889271 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:19:56.998039 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2baaa-8561-4f45-ad97-b3a342889271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:19:57.001627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2baaa-8561-4f45-ad97-b3a342889271 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:19:57.033488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-49e2baaa-8561-4f45-ad97-b3a342889271 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:19:57.033488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:19:57.033488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:19:57.033488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:19:57.033488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:19:57.033488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:19:57.033488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:19:57.033488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:19:57.033488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:19:57.033488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:19:57.033488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:19:57.033488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:19:57.033488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:19:57.033488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:19:57.033488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:19:57.035058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:19:57.035058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:19:57.035058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:19:57.035058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:19:57.035058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:19:57.035058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:19:57.035058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:19:57.035058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:19:57.035058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:19:57.035058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:19:57.035058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:19:57.035058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:19:57.035058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:19:57.035058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:19:57.035058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:19:57.035058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:19:57.035058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:19:57.045608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:19:57.045608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:19:57.045608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:19:57.045608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:19:57.045608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 598/1203] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:19:57.046031 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9067a8-5887-4597-9d2f-b16f363419dd None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:19:57.203960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9067a8-5887-4597-9d2f-b16f363419dd None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:19:57.256479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:19:57.266046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000c730-b21e-4e5b-b62e-b8d391c37fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:19:57.267033 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000c730-b21e-4e5b-b62e-b8d391c37fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:19:57.267377 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000c730-b21e-4e5b-b62e-b8d391c37fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:19:57.267641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000c730-b21e-4e5b-b62e-b8d391c37fd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:19:57.270119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000c730-b21e-4e5b-b62e-b8d391c37fd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:19:57.270432 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000c730-b21e-4e5b-b62e-b8d391c37fd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:19:57.272539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000c730-b21e-4e5b-b62e-b8d391c37fd8 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:19:57.294886 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 606/1204] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:19:57.302834 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2baaa-8561-4f45-ad97-b3a342889271 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:19:57.387057 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:19:57.452145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2baaa-8561-4f45-ad97-b3a342889271 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:19:57.504907 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a207479-6305-4404-9a3c-1f7e23fb5686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:19:57.505654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a207479-6305-4404-9a3c-1f7e23fb5686 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:19:57.505955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a207479-6305-4404-9a3c-1f7e23fb5686 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:19:57.506281 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a207479-6305-4404-9a3c-1f7e23fb5686 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:19:57.507069 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a207479-6305-4404-9a3c-1f7e23fb5686 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:19:57.508713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a207479-6305-4404-9a3c-1f7e23fb5686 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:19:57.534409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 599/1205] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 06:19:57 2022] POST /identity/v3/users => generated 251 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:19:59.948699 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168b10c-01ac-466a-8b8e-61021afc49d1 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:19:59.949256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168b10c-01ac-466a-8b8e-61021afc49d1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:19:59.949256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168b10c-01ac-466a-8b8e-61021afc49d1 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:19:59.950859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 607/1206] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:19:59.957226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f22d3a-64d5-40bc-a796-c35054f44728 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:19:59.957755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f22d3a-64d5-40bc-a796-c35054f44728 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:19:59.957856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f22d3a-64d5-40bc-a796-c35054f44728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:19:59.978836 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d3f22d3a-64d5-40bc-a796-c35054f44728 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:19:59.980021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d3f22d3a-64d5-40bc-a796-c35054f44728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:20:00.061947 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 600/1207] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:19:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:20:00.085259 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168b10c-01ac-466a-8b8e-61021afc49d1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:00.217395 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168b10c-01ac-466a-8b8e-61021afc49d1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:00.261724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ac468-2636-4dee-8fc5-4faada6cb998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:00.262693 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ac468-2636-4dee-8fc5-4faada6cb998 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:00.263083 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ac468-2636-4dee-8fc5-4faada6cb998 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:00.263465 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ac468-2636-4dee-8fc5-4faada6cb998 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:00.276403 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ac468-2636-4dee-8fc5-4faada6cb998 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:20:00.284927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-410ac468-2636-4dee-8fc5-4faada6cb998 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:20:00.284927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:00.284927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:00.284927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:00.284927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:00.284927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:00.284927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:00.284927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:00.284927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:00.284927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:00.284927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:00.284927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:00.284927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:20:00.284927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:20:00.284927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:20:00.284927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:20:00.294277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:20:00.294277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:20:00.294277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:20:00.294277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:20:00.294277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:00.294277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:00.294277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:00.294277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:00.294277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:00.294277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:00.294277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:00.294277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:00.294277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:00.294277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:00.294277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:00.294277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:00.296137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:00.296137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:00.296137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:00.296137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:00.296137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:20:00.296137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:20:00.296137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:20:00.296137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:20:00.296137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:20:00.296137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:20:00.296137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:20:00.296137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:00.296137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 608/1208] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:20:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:00.301844 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f22d3a-64d5-40bc-a796-c35054f44728 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:00.355838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:00.423213 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f22d3a-64d5-40bc-a796-c35054f44728 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:00.461234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd74b8-e8d8-494e-8700-ea3f7302013b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:00.461981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd74b8-e8d8-494e-8700-ea3f7302013b None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:00.462267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd74b8-e8d8-494e-8700-ea3f7302013b None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:00.462541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd74b8-e8d8-494e-8700-ea3f7302013b None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:00.463302 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd74b8-e8d8-494e-8700-ea3f7302013b None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:00.463610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd74b8-e8d8-494e-8700-ea3f7302013b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:00.465400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd74b8-e8d8-494e-8700-ea3f7302013b None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:00.477208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 601/1209] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:20:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:00.482764 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ac468-2636-4dee-8fc5-4faada6cb998 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:00.543127 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:00.599832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ac468-2636-4dee-8fc5-4faada6cb998 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:00.639234 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6f5aa-986f-487b-a4c4-81b5cea20b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:00.640386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6f5aa-986f-487b-a4c4-81b5cea20b0a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:00.640652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6f5aa-986f-487b-a4c4-81b5cea20b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:00.640933 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6f5aa-986f-487b-a4c4-81b5cea20b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:00.654599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6f5aa-986f-487b-a4c4-81b5cea20b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:00.657162 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6f5aa-986f-487b-a4c4-81b5cea20b0a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:00.677299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-80e6f5aa-986f-487b-a4c4-81b5cea20b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:00.677299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:00.677299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:00.677299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:00.677299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:00.677299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:00.677299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:00.677299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:00.677299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:00.677299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:00.677299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:00.677299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:00.677299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:20:00.677299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:20:00.677299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:20:00.681551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:20:00.681551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:00.681551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:00.681551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:00.681551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:00.681551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:00.681551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:00.681551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:00.681551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:00.681551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:00.681551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:00.681551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:00.681551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:00.681551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:00.681551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:00.681551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:00.681551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:00.682579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:20:00.682579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:20:00.682579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:00.682579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:00.682579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 609/1210] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:00.684232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd74b8-e8d8-494e-8700-ea3f7302013b None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:00.813386 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:00.818698 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd74b8-e8d8-494e-8700-ea3f7302013b None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:00.866988 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fc237-a5d0-4037-a7b7-f27ca595ac19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:00.867827 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fc237-a5d0-4037-a7b7-f27ca595ac19 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:00.868200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fc237-a5d0-4037-a7b7-f27ca595ac19 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:00.868553 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fc237-a5d0-4037-a7b7-f27ca595ac19 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:00.869385 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fc237-a5d0-4037-a7b7-f27ca595ac19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:00.869783 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fc237-a5d0-4037-a7b7-f27ca595ac19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:00.871218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fc237-a5d0-4037-a7b7-f27ca595ac19 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:00.885354 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 602/1211] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:20:00.890375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6f5aa-986f-487b-a4c4-81b5cea20b0a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:01.029665 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6f5aa-986f-487b-a4c4-81b5cea20b0a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:01.031763 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:01.069571 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5af0c-3522-4bca-bfdf-f9de739a954a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:01.070342 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5af0c-3522-4bca-bfdf-f9de739a954a None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:01.070592 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5af0c-3522-4bca-bfdf-f9de739a954a None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:01.070866 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5af0c-3522-4bca-bfdf-f9de739a954a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:01.078479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a5af0c-3522-4bca-bfdf-f9de739a954a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:01.079936 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a5af0c-3522-4bca-bfdf-f9de739a954a None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:01.097686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-80a5af0c-3522-4bca-bfdf-f9de739a954a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 06:20:01.097686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:01.097686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:01.097686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:01.097686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:01.097686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:01.097686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:01.097686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:01.097686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:01.097686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:01.097686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:01.097686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:01.097686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:20:01.097686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:20:01.097686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:20:01.097686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:01.099156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:01.100459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:01.100459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:01.100459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:20:01.100459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:20:01.100459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:20:01.100459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:20:01.100459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:20:01.100459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:20:01.100459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 06:20:01.100459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:01.100459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 610/1212] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:20:01.105946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fc237-a5d0-4037-a7b7-f27ca595ac19 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:01.227595 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fc237-a5d0-4037-a7b7-f27ca595ac19 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:01.278134 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b522fa-b18a-4499-be04-2576a037d526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:01.279099 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b522fa-b18a-4499-be04-2576a037d526 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:01.279512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b522fa-b18a-4499-be04-2576a037d526 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:01.279969 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b522fa-b18a-4499-be04-2576a037d526 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:01.281243 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b522fa-b18a-4499-be04-2576a037d526 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:01.281604 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b522fa-b18a-4499-be04-2576a037d526 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:01.283373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b522fa-b18a-4499-be04-2576a037d526 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:01.308021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 603/1213] 10.222.0.65 () {56 vars in 1143 bytes} [Thu May 19 06:20:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:01.316748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5af0c-3522-4bca-bfdf-f9de739a954a None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:01.462341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5af0c-3522-4bca-bfdf-f9de739a954a None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:01.503539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958b019-0a5a-4dfb-9326-d2072a857002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:01.504314 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958b019-0a5a-4dfb-9326-d2072a857002 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:01.504674 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958b019-0a5a-4dfb-9326-d2072a857002 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:01.504960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958b019-0a5a-4dfb-9326-d2072a857002 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:01.541746 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:01.589001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958b019-0a5a-4dfb-9326-d2072a857002 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:01.589432 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958b019-0a5a-4dfb-9326-d2072a857002 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:01.713300 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 611/1214] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:20:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:20:01.722873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b522fa-b18a-4499-be04-2576a037d526 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:01.746739 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:01.847362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b522fa-b18a-4499-be04-2576a037d526 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:01.888298 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159f455-eb2e-4304-bd97-db00ce812361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:01.889103 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159f455-eb2e-4304-bd97-db00ce812361 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:01.889418 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159f455-eb2e-4304-bd97-db00ce812361 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:01.889701 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159f455-eb2e-4304-bd97-db00ce812361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:01.903305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159f455-eb2e-4304-bd97-db00ce812361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:01.905369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159f455-eb2e-4304-bd97-db00ce812361 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:01.927072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f159f455-eb2e-4304-bd97-db00ce812361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:01.927072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:01.927072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:01.927072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:01.927072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:01.927072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:01.927072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:01.927072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:01.927072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:01.927072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:01.927072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:01.927072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:01.927072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:20:01.927072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:20:01.927072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:20:01.928541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:20:01.928541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:01.928541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:01.928541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:01.928541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:01.928541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:01.928541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:01.928541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:01.928541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:20:01.928541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:01.928541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:01.928541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:01.928541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:01.928541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:01.928541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:01.928541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:01.928541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:01.929980 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:20:01.929980 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:20:01.929980 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:01.929980 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:20:01.939445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 604/1215] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:01.941792 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958b019-0a5a-4dfb-9326-d2072a857002 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:02.040944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:02.094976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958b019-0a5a-4dfb-9326-d2072a857002 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:02.134956 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a2700-a45c-4eb3-a777-9faf46ae1ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:02.135691 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a2700-a45c-4eb3-a777-9faf46ae1ffb None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:02.136009 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a2700-a45c-4eb3-a777-9faf46ae1ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:02.136457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a2700-a45c-4eb3-a777-9faf46ae1ffb None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:02.137235 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a2700-a45c-4eb3-a777-9faf46ae1ffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:02.137658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a2700-a45c-4eb3-a777-9faf46ae1ffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:02.139181 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a2700-a45c-4eb3-a777-9faf46ae1ffb None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:02.162991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 612/1216] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:02.164467 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159f455-eb2e-4304-bd97-db00ce812361 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:02.259144 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:02.302398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159f455-eb2e-4304-bd97-db00ce812361 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:02.349234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66242ad-9495-49c4-ae96-b9be3ed6d8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:02.350413 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66242ad-9495-49c4-ae96-b9be3ed6d8ef None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:02.350767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66242ad-9495-49c4-ae96-b9be3ed6d8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:02.351071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66242ad-9495-49c4-ae96-b9be3ed6d8ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:02.360497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66242ad-9495-49c4-ae96-b9be3ed6d8ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:02.362423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66242ad-9495-49c4-ae96-b9be3ed6d8ef None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:02.377130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-a66242ad-9495-49c4-ae96-b9be3ed6d8ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:20:02.377130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:02.377130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:02.377130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:02.377130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:02.377130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:02.377130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:02.377130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:02.377130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:02.377130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:02.377130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:02.377130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:02.377130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:20:02.377130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:20:02.377130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:20:02.378921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:20:02.378921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:02.378921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:02.378921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:02.378921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:02.378921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:02.378921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:02.378921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:02.378921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:20:02.378921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:02.378921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:02.378921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:02.378921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:02.378921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:02.378921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:02.378921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:02.378921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:02.380077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:20:02.380077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:20:02.380077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:20:02.380077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:20:02.380077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:20:02.380077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:20:02.380077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:20:02.380077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:20:02.380077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 605/1217] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:20:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:02.387605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a2700-a45c-4eb3-a777-9faf46ae1ffb None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:02.519302 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a2700-a45c-4eb3-a777-9faf46ae1ffb None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:02.550992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:02.567733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebda100-72ef-48cb-9d45-fcd83f47b705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:02.568490 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebda100-72ef-48cb-9d45-fcd83f47b705 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:02.568872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebda100-72ef-48cb-9d45-fcd83f47b705 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:02.569179 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebda100-72ef-48cb-9d45-fcd83f47b705 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:02.570037 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebda100-72ef-48cb-9d45-fcd83f47b705 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:02.570360 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebda100-72ef-48cb-9d45-fcd83f47b705 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:02.572031 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebda100-72ef-48cb-9d45-fcd83f47b705 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:02.589538 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 613/1218] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:20:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:02.595815 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66242ad-9495-49c4-ae96-b9be3ed6d8ef None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:02.712059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66242ad-9495-49c4-ae96-b9be3ed6d8ef None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:02.743814 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:02.750296 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134ccca-b894-4fc9-81bb-f226ca87e6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:02.751043 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134ccca-b894-4fc9-81bb-f226ca87e6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:02.751397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134ccca-b894-4fc9-81bb-f226ca87e6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:02.751741 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134ccca-b894-4fc9-81bb-f226ca87e6b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:02.753719 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134ccca-b894-4fc9-81bb-f226ca87e6b1 None admin] RBAC: Adding query filter params (role.id=5cc9ca1bc48b4d749d21d7111b14e187, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=1b76876d821e4bbbb8d5cad5bd26f559) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:02.753719 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134ccca-b894-4fc9-81bb-f226ca87e6b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:02.755317 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134ccca-b894-4fc9-81bb-f226ca87e6b1 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:02.769657 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 606/1219] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:20:02 2022] GET /identity/v3/role_assignments?user.id=1b76876d821e4bbbb8d5cad5bd26f559&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=5cc9ca1bc48b4d749d21d7111b14e187 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:20:05.002580 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98697a9b-b0a8-49f9-accb-c23762316ef1 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:05.003248 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98697a9b-b0a8-49f9-accb-c23762316ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:05.003874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98697a9b-b0a8-49f9-accb-c23762316ef1 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:05.006768 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 614/1220] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:20:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:20:05.014223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235533ac-e0d2-4aa0-8e4e-07e1169b5701 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:05.014855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235533ac-e0d2-4aa0-8e4e-07e1169b5701 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:05.015241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235533ac-e0d2-4aa0-8e4e-07e1169b5701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:05.035395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-235533ac-e0d2-4aa0-8e4e-07e1169b5701 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:20:05.036596 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-235533ac-e0d2-4aa0-8e4e-07e1169b5701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:20:05.121056 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 607/1221] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:20:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:20:05.147291 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98697a9b-b0a8-49f9-accb-c23762316ef1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:05.270361 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98697a9b-b0a8-49f9-accb-c23762316ef1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:05.317785 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029ec8c-1862-460f-88df-3513b81159c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:05.321310 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029ec8c-1862-460f-88df-3513b81159c6 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:05.321310 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029ec8c-1862-460f-88df-3513b81159c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:05.321310 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029ec8c-1862-460f-88df-3513b81159c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:05.343628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029ec8c-1862-460f-88df-3513b81159c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:20:05.351632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-2029ec8c-1862-460f-88df-3513b81159c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:20:05.351632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:05.351632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:05.351632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:05.351632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:05.351632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:05.351632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:05.351632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:05.351632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:05.351632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:05.351632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:05.351632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:05.351632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:20:05.351632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:20:05.351632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:20:05.351632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:20:05.352724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:20:05.352724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:20:05.352724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:20:05.352724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:20:05.352724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:05.352724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:05.352724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:05.352724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:05.352724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:05.352724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:05.352724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:05.352724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:05.352724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:05.352724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:05.352724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:05.352724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:05.353623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:05.353623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:05.353623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:05.353623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:05.353623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:20:05.353623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:20:05.353623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:20:05.353623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:20:05.353623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:20:05.353623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:20:05.353623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:20:05.353623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:05.358200 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 615/1222] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:20:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:05.359537 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235533ac-e0d2-4aa0-8e4e-07e1169b5701 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:05.488179 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235533ac-e0d2-4aa0-8e4e-07e1169b5701 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:05.517091 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:05.529831 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1899145-d9e6-4b2b-9549-595cca2c7016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:05.530447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1899145-d9e6-4b2b-9549-595cca2c7016 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:05.530731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1899145-d9e6-4b2b-9549-595cca2c7016 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:05.531008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1899145-d9e6-4b2b-9549-595cca2c7016 None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:05.531761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1899145-d9e6-4b2b-9549-595cca2c7016 None admin] RBAC: Adding query filter params (name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:05.532066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1899145-d9e6-4b2b-9549-595cca2c7016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:05.533482 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1899145-d9e6-4b2b-9549-595cca2c7016 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:05.541720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 608/1223] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:20:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:20:05.547624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029ec8c-1862-460f-88df-3513b81159c6 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:05.594195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:05.655662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029ec8c-1862-460f-88df-3513b81159c6 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:05.692712 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779a602-bfa8-4f9c-a999-16c2dc7f614c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:05.693622 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779a602-bfa8-4f9c-a999-16c2dc7f614c None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:05.694000 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779a602-bfa8-4f9c-a999-16c2dc7f614c None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:05.694292 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779a602-bfa8-4f9c-a999-16c2dc7f614c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:05.706505 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779a602-bfa8-4f9c-a999-16c2dc7f614c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:05.707951 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779a602-bfa8-4f9c-a999-16c2dc7f614c None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:05.727684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-4779a602-bfa8-4f9c-a999-16c2dc7f614c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:05.727684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:05.727684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:05.727684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:05.727684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:05.727684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:05.727684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:05.727684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:05.727684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:05.727684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:05.727684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:05.727684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:05.727684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:20:05.727684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:20:05.727684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:20:05.728845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:20:05.728845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:05.728845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:05.728845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:05.728845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:05.728845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:05.728845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:05.728845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:05.728845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:05.728845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:05.728845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:05.728845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:05.728845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:05.728845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:05.728845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:05.728845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:05.728845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:05.734808 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:20:05.734808 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:20:05.734808 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:05.734808 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:05.734808 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 616/1224] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:20:05.735179 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1899145-d9e6-4b2b-9549-595cca2c7016 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:05.851125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1899145-d9e6-4b2b-9549-595cca2c7016 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:05.887852 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4ed9e-73b9-45af-acc2-e1a3e3d7394c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:05.888503 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4ed9e-73b9-45af-acc2-e1a3e3d7394c None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:05.888828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4ed9e-73b9-45af-acc2-e1a3e3d7394c None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:05.889137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4ed9e-73b9-45af-acc2-e1a3e3d7394c None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:05.889865 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4ed9e-73b9-45af-acc2-e1a3e3d7394c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:05.890174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4ed9e-73b9-45af-acc2-e1a3e3d7394c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:05.891553 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4ed9e-73b9-45af-acc2-e1a3e3d7394c None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:05.905154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 609/1225] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:05.910924 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779a602-bfa8-4f9c-a999-16c2dc7f614c None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:06.018969 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:06.026828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779a602-bfa8-4f9c-a999-16c2dc7f614c None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:06.072518 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65227d3c-566c-4fa9-a2e6-a05c160e6877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:06.073361 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65227d3c-566c-4fa9-a2e6-a05c160e6877 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:06.073651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65227d3c-566c-4fa9-a2e6-a05c160e6877 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:06.073930 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65227d3c-566c-4fa9-a2e6-a05c160e6877 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:06.083105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65227d3c-566c-4fa9-a2e6-a05c160e6877 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:06.084818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65227d3c-566c-4fa9-a2e6-a05c160e6877 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:06.100865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-65227d3c-566c-4fa9-a2e6-a05c160e6877 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 06:20:06.100865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:06.100865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:06.100865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:06.100865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:06.100865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:06.100865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:06.100865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:06.100865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:06.100865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:06.100865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:06.100865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:06.100865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:20:06.100865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:20:06.100865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:20:06.100865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:06.102146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:06.103365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:06.103365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:06.103365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:20:06.103365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:20:06.103365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:20:06.103365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:20:06.103365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:20:06.103365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:20:06.103365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 06:20:06.103365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:06.103365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 617/1226] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:06.108215 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4ed9e-73b9-45af-acc2-e1a3e3d7394c None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:06.161855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:06.220221 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4ed9e-73b9-45af-acc2-e1a3e3d7394c None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:06.269695 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af0ad4-2bbf-49d4-9ffd-3fa6142b3623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:06.270292 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af0ad4-2bbf-49d4-9ffd-3fa6142b3623 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:06.270576 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af0ad4-2bbf-49d4-9ffd-3fa6142b3623 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:06.270858 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af0ad4-2bbf-49d4-9ffd-3fa6142b3623 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:06.271741 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af0ad4-2bbf-49d4-9ffd-3fa6142b3623 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:06.272015 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af0ad4-2bbf-49d4-9ffd-3fa6142b3623 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:06.273475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af0ad4-2bbf-49d4-9ffd-3fa6142b3623 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:06.291840 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 610/1227] 10.222.0.65 () {56 vars in 1143 bytes} [Thu May 19 06:20:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:06.297541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65227d3c-566c-4fa9-a2e6-a05c160e6877 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:06.411765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65227d3c-566c-4fa9-a2e6-a05c160e6877 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:06.454838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05bf7e-bf1d-432d-af1d-7d0f36a7f3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:06.455674 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05bf7e-bf1d-432d-af1d-7d0f36a7f3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:06.456041 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05bf7e-bf1d-432d-af1d-7d0f36a7f3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:06.456322 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05bf7e-bf1d-432d-af1d-7d0f36a7f3f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:06.467415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd05bf7e-bf1d-432d-af1d-7d0f36a7f3f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:06.469111 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd05bf7e-bf1d-432d-af1d-7d0f36a7f3f9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:06.488014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-dd05bf7e-bf1d-432d-af1d-7d0f36a7f3f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:06.488014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:06.488014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:06.488014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:06.488014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:06.488014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:06.488014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:06.488014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:06.488014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:06.488014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:06.488014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:06.488014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:06.488014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:20:06.488014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:20:06.488014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:20:06.489188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:20:06.489188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:06.489188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:06.489188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:06.489188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:06.489188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:06.489188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:06.489188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:06.489188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:06.489188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:06.489188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:06.489188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:06.489188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:06.489188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:06.489188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:06.489188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:06.489188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:06.490188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:20:06.490188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:20:06.490188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:06.490188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:06.490188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 618/1228] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:06.495137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af0ad4-2bbf-49d4-9ffd-3fa6142b3623 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:06.606005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af0ad4-2bbf-49d4-9ffd-3fa6142b3623 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:06.608384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:06.642330 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebd104-cc59-47cf-986f-534057601e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:06.643573 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebd104-cc59-47cf-986f-534057601e57 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:06.643980 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebd104-cc59-47cf-986f-534057601e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:06.644255 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebd104-cc59-47cf-986f-534057601e57 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:06.644978 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cebd104-cc59-47cf-986f-534057601e57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:06.645313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cebd104-cc59-47cf-986f-534057601e57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:06.646690 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cebd104-cc59-47cf-986f-534057601e57 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:06.658604 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 611/1229] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:06.664300 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05bf7e-bf1d-432d-af1d-7d0f36a7f3f9 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:06.674033 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:06.785267 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05bf7e-bf1d-432d-af1d-7d0f36a7f3f9 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:06.820132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3d301-c848-42c2-afcb-5fb8ee89b281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:06.820939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3d301-c848-42c2-afcb-5fb8ee89b281 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:06.821218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3d301-c848-42c2-afcb-5fb8ee89b281 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:06.821523 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3d301-c848-42c2-afcb-5fb8ee89b281 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:06.828502 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3d301-c848-42c2-afcb-5fb8ee89b281 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:06.830011 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3d301-c848-42c2-afcb-5fb8ee89b281 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:06.844767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-37d3d301-c848-42c2-afcb-5fb8ee89b281 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:20:06.844767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:06.844767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:06.844767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:06.844767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:06.844767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:06.844767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:06.844767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:06.844767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:06.844767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:06.844767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:06.844767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:06.844767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:20:06.844767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:20:06.844767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:20:06.846447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:20:06.846447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:06.846447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:06.846447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:06.846447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:06.846447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:06.846447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:06.846447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:06.846447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:06.846447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:06.846447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:06.846447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:06.846447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:06.846447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:06.846447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:06.846447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:06.846447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:06.852277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:20:06.852277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:20:06.852277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:20:06.852277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:20:06.852277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:20:06.852277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:20:06.852277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:20:06.852277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:06.852277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 619/1230] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:20:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:06.853242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebd104-cc59-47cf-986f-534057601e57 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:06.972980 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebd104-cc59-47cf-986f-534057601e57 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:07.013445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190c848-4ca7-4d01-b174-d7cd7c7a8dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:07.014280 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190c848-4ca7-4d01-b174-d7cd7c7a8dad None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:07.014663 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190c848-4ca7-4d01-b174-d7cd7c7a8dad None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:07.015066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190c848-4ca7-4d01-b174-d7cd7c7a8dad None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:07.015951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190c848-4ca7-4d01-b174-d7cd7c7a8dad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:07.016340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190c848-4ca7-4d01-b174-d7cd7c7a8dad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:07.018323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190c848-4ca7-4d01-b174-d7cd7c7a8dad None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:07.031492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 612/1231] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:20:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:07.038535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3d301-c848-42c2-afcb-5fb8ee89b281 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:07.106228 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:07.145870 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3d301-c848-42c2-afcb-5fb8ee89b281 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:07.183913 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebb5ff-c296-4f7a-bf4d-7c01b4fb7c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:07.184834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebb5ff-c296-4f7a-bf4d-7c01b4fb7c1f None admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:07.185093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebb5ff-c296-4f7a-bf4d-7c01b4fb7c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:07.185393 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebb5ff-c296-4f7a-bf4d-7c01b4fb7c1f None admin] PATH_INFO: `/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/1b76876d821e4bbbb8d5cad5bd26f559/roles/5cc9ca1bc48b4d749d21d7111b14e187` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:07.200742 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ebb5ff-c296-4f7a-bf4d-7c01b4fb7c1f None admin] RBAC: Authorizing `identity:create_grant(project_id=a3d399dc95424b848b736c0d0298d9a9, user_id=1b76876d821e4bbbb8d5cad5bd26f559, role_id=5cc9ca1bc48b4d749d21d7111b14e187)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:07.202238 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ebb5ff-c296-4f7a-bf4d-7c01b4fb7c1f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:07.219691 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 620/1232] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:20:07 2022] PUT /identity/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/1b76876d821e4bbbb8d5cad5bd26f559/roles/5cc9ca1bc48b4d749d21d7111b14e187 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:20:09.824632 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f567b-d6b7-4266-be54-b195c3f4988a None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:09.825164 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f567b-d6b7-4266-be54-b195c3f4988a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:09.825164 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f567b-d6b7-4266-be54-b195c3f4988a None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:09.826765 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 613/1233] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:20:09.832508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880dd2e-9a44-4a5f-a194-b5afcb57143f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:09.832974 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880dd2e-9a44-4a5f-a194-b5afcb57143f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:09.833066 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880dd2e-9a44-4a5f-a194-b5afcb57143f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:09.850628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0880dd2e-9a44-4a5f-a194-b5afcb57143f None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:20:09.851838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0880dd2e-9a44-4a5f-a194-b5afcb57143f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:20:09.923834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 621/1234] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:20:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 06:20:09.948688 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f567b-d6b7-4266-be54-b195c3f4988a None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:09.976798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:10.083165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f567b-d6b7-4266-be54-b195c3f4988a None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:10.119816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6883fb-072b-4a36-b77c-a3607dca35f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:10.120651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6883fb-072b-4a36-b77c-a3607dca35f2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:10.120936 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6883fb-072b-4a36-b77c-a3607dca35f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:10.121212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6883fb-072b-4a36-b77c-a3607dca35f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:10.132935 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6883fb-072b-4a36-b77c-a3607dca35f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:20:10.143118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-be6883fb-072b-4a36-b77c-a3607dca35f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 06:20:10.143118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:10.143118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:10.143118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:10.143118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:10.143118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:10.143118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:10.143118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:10.143118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:10.143118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:10.143118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:10.143118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:10.143118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:20:10.143118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:20:10.143118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:20:10.143118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:20:10.144534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:20:10.144534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:20:10.144534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:20:10.144534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:20:10.144534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:10.144534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:10.144534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:10.144534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:10.144534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:10.144534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:10.144534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:10.144534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:20:10.144534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:10.144534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:10.144534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:10.144534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:10.145438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:10.145438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:10.145438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:10.145438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:10.145438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:20:10.145438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:20:10.145438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:20:10.145438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:20:10.145438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:20:10.145438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:20:10.145438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 06:20:10.145438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:20:10.145438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 614/1235] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 06:20:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:10.150552 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880dd2e-9a44-4a5f-a194-b5afcb57143f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:10.200991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:10.258775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880dd2e-9a44-4a5f-a194-b5afcb57143f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:10.295941 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c4d3b-80a7-4378-98e4-56c300a88708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:10.296619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c4d3b-80a7-4378-98e4-56c300a88708 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:10.296905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c4d3b-80a7-4378-98e4-56c300a88708 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:10.297185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c4d3b-80a7-4378-98e4-56c300a88708 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:10.297960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c4d3b-80a7-4378-98e4-56c300a88708 None admin] RBAC: Adding query filter params (name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:10.298268 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c4d3b-80a7-4378-98e4-56c300a88708 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:10.299733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c4d3b-80a7-4378-98e4-56c300a88708 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:10.308173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 622/1236] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 06:20:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:10.314024 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6883fb-072b-4a36-b77c-a3607dca35f2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:10.412193 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:10.433086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6883fb-072b-4a36-b77c-a3607dca35f2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:10.478754 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8664e3-a652-40dc-938f-e37f134414f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:10.479560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8664e3-a652-40dc-938f-e37f134414f2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:10.479850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8664e3-a652-40dc-938f-e37f134414f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:10.480131 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8664e3-a652-40dc-938f-e37f134414f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:10.493856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8664e3-a652-40dc-938f-e37f134414f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:10.495333 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8664e3-a652-40dc-938f-e37f134414f2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:10.514128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-3d8664e3-a652-40dc-938f-e37f134414f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:10.514128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:10.514128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:10.514128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:10.514128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:10.514128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:10.514128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:10.514128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:10.514128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:10.514128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:10.514128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:10.514128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:10.514128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:20:10.514128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:20:10.514128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:20:10.515327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:20:10.515327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:10.515327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:10.515327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:10.515327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:10.515327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:10.515327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:10.515327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:10.515327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:20:10.515327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:10.515327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:10.515327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:10.515327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:10.515327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:10.515327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:10.515327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:10.515327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:10.516486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:20:10.516486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:20:10.516486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:10.516486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:20:10.516486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 615/1237] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:10.523869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c4d3b-80a7-4378-98e4-56c300a88708 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:10.646131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c4d3b-80a7-4378-98e4-56c300a88708 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:10.658000 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:10.691980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed8bb9-5d5d-4e03-ac99-d61d1ffb0360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:10.692699 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed8bb9-5d5d-4e03-ac99-d61d1ffb0360 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:10.693081 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed8bb9-5d5d-4e03-ac99-d61d1ffb0360 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:10.693241 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed8bb9-5d5d-4e03-ac99-d61d1ffb0360 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:10.693971 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed8bb9-5d5d-4e03-ac99-d61d1ffb0360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:10.694278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed8bb9-5d5d-4e03-ac99-d61d1ffb0360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:10.696253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed8bb9-5d5d-4e03-ac99-d61d1ffb0360 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:10.708774 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 623/1238] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:10.716040 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8664e3-a652-40dc-938f-e37f134414f2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:10.847276 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8664e3-a652-40dc-938f-e37f134414f2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:10.898238 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e141fd2-39bf-4aeb-9d7b-8f7c096ecc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:10.899046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e141fd2-39bf-4aeb-9d7b-8f7c096ecc1d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:10.899331 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e141fd2-39bf-4aeb-9d7b-8f7c096ecc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:10.899606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e141fd2-39bf-4aeb-9d7b-8f7c096ecc1d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:10.902973 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:10.909944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e141fd2-39bf-4aeb-9d7b-8f7c096ecc1d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:10.911928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e141fd2-39bf-4aeb-9d7b-8f7c096ecc1d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:10.926654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-0e141fd2-39bf-4aeb-9d7b-8f7c096ecc1d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 06:20:10.926654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:10.926654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:10.926654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:10.926654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:10.926654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:10.926654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:10.926654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:10.926654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:10.926654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:10.926654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:10.926654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:10.926654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:20:10.926654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:20:10.926654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:20:10.926654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:10.927888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:10.929063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:10.929063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:10.929063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:20:10.929063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:20:10.929063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:20:10.929063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:20:10.929063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:20:10.929063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:20:10.929063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 06:20:10.929063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:20:10.929063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 616/1239] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:10.937090 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed8bb9-5d5d-4e03-ac99-d61d1ffb0360 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:11.061114 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed8bb9-5d5d-4e03-ac99-d61d1ffb0360 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:11.112368 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6826851-4e5d-4aad-b05f-38462d7d39f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:11.113071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6826851-4e5d-4aad-b05f-38462d7d39f5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:11.113488 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6826851-4e5d-4aad-b05f-38462d7d39f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:11.113765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6826851-4e5d-4aad-b05f-38462d7d39f5 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:11.114732 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6826851-4e5d-4aad-b05f-38462d7d39f5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:11.114954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6826851-4e5d-4aad-b05f-38462d7d39f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:11.116475 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6826851-4e5d-4aad-b05f-38462d7d39f5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:11.137036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 624/1240] 10.222.0.65 () {56 vars in 1143 bytes} [Thu May 19 06:20:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:20:11.143670 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e141fd2-39bf-4aeb-9d7b-8f7c096ecc1d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:11.267525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e141fd2-39bf-4aeb-9d7b-8f7c096ecc1d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:11.307379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a3e4c-9c50-451f-8da8-26f3cdc3b8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:11.308028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a3e4c-9c50-451f-8da8-26f3cdc3b8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:11.308319 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a3e4c-9c50-451f-8da8-26f3cdc3b8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:11.308612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a3e4c-9c50-451f-8da8-26f3cdc3b8a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:11.387269 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:11.396129 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a3e4c-9c50-451f-8da8-26f3cdc3b8a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:11.398227 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a3e4c-9c50-451f-8da8-26f3cdc3b8a3 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:11.529484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 617/1241] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:20:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 06:20:11.538043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6826851-4e5d-4aad-b05f-38462d7d39f5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:11.572746 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:11.659898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6826851-4e5d-4aad-b05f-38462d7d39f5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:11.710331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b1190-726d-44ae-b8b5-29f14d89710f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:11.711179 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b1190-726d-44ae-b8b5-29f14d89710f None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:11.711467 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b1190-726d-44ae-b8b5-29f14d89710f None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:11.711904 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b1190-726d-44ae-b8b5-29f14d89710f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:11.725600 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b1190-726d-44ae-b8b5-29f14d89710f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:11.727378 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b1190-726d-44ae-b8b5-29f14d89710f None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:11.748375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-d55b1190-726d-44ae-b8b5-29f14d89710f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:11.748375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:11.748375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:11.748375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:11.748375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:11.748375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:11.748375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:11.748375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:11.748375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:11.748375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:11.748375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:11.748375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:11.748375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:20:11.748375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:20:11.748375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:20:11.749720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:20:11.749720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:11.749720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:11.749720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:11.749720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:11.749720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:11.749720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:11.749720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:11.749720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:11.749720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:11.749720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:11.749720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:11.749720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:11.749720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:11.749720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:11.749720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:11.749720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:11.750877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:20:11.750877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:20:11.750877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:11.750877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:11.750877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 625/1242] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:11.756683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a3e4c-9c50-451f-8da8-26f3cdc3b8a3 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:11.867332 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:11.873507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a3e4c-9c50-451f-8da8-26f3cdc3b8a3 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:11.914551 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09672a-85b6-484d-a136-694ea0b0a243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:11.915212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09672a-85b6-484d-a136-694ea0b0a243 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:11.915494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09672a-85b6-484d-a136-694ea0b0a243 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:11.915771 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09672a-85b6-484d-a136-694ea0b0a243 None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:11.916512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09672a-85b6-484d-a136-694ea0b0a243 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:11.916814 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09672a-85b6-484d-a136-694ea0b0a243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:11.918223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09672a-85b6-484d-a136-694ea0b0a243 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:11.931399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 618/1243] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:11.937114 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b1190-726d-44ae-b8b5-29f14d89710f None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:12.033770 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:12.065522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b1190-726d-44ae-b8b5-29f14d89710f None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:12.108800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195b71a-55d3-4334-92a3-5e8d19cf9853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:12.109717 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195b71a-55d3-4334-92a3-5e8d19cf9853 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:12.110009 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195b71a-55d3-4334-92a3-5e8d19cf9853 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:12.110284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195b71a-55d3-4334-92a3-5e8d19cf9853 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:12.118416 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195b71a-55d3-4334-92a3-5e8d19cf9853 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:12.119728 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195b71a-55d3-4334-92a3-5e8d19cf9853 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:12.135706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-2195b71a-55d3-4334-92a3-5e8d19cf9853 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:20:12.135706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:12.135706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:12.135706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:12.135706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:12.135706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:12.135706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:12.135706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:12.135706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:12.135706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:12.135706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:12.135706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:12.135706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:20:12.135706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:20:12.135706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:20:12.136914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:20:12.136914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:12.136914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:12.136914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:12.136914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:12.136914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:12.136914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:12.136914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:12.136914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:12.136914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:12.136914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:12.136914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:12.136914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:12.136914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:12.136914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:12.136914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:12.136914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:12.137975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:20:12.137975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:20:12.137975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:20:12.137975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:20:12.137975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:20:12.137975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:20:12.137975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:20:12.137975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:12.142963 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 626/1244] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:20:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:20:12.143994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09672a-85b6-484d-a136-694ea0b0a243 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:12.274769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09672a-85b6-484d-a136-694ea0b0a243 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:12.316731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb34ca1-652b-416c-8b77-5647668a89d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:12.317744 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb34ca1-652b-416c-8b77-5647668a89d9 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:12.318150 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb34ca1-652b-416c-8b77-5647668a89d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:12.318540 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb34ca1-652b-416c-8b77-5647668a89d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:12.321389 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb34ca1-652b-416c-8b77-5647668a89d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:12.321389 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb34ca1-652b-416c-8b77-5647668a89d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:12.321389 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb34ca1-652b-416c-8b77-5647668a89d9 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:12.340524 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 619/1245] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:20:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:20:12.347063 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195b71a-55d3-4334-92a3-5e8d19cf9853 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:12.490855 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195b71a-55d3-4334-92a3-5e8d19cf9853 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:12.539163 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866feee9-707f-43e0-bb38-d31564c81857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:12.540126 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866feee9-707f-43e0-bb38-d31564c81857 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:12.540511 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866feee9-707f-43e0-bb38-d31564c81857 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:12.540872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866feee9-707f-43e0-bb38-d31564c81857 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:12.541927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866feee9-707f-43e0-bb38-d31564c81857 None admin] RBAC: Adding query filter params (role.id=5cc9ca1bc48b4d749d21d7111b14e187, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=1b76876d821e4bbbb8d5cad5bd26f559) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:12.542353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866feee9-707f-43e0-bb38-d31564c81857 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:12.543946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866feee9-707f-43e0-bb38-d31564c81857 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:12.551451 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 627/1246] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:20:12 2022] GET /identity/v3/role_assignments?user.id=1b76876d821e4bbbb8d5cad5bd26f559&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=5cc9ca1bc48b4d749d21d7111b14e187 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:15.060927 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac892589-d8c0-4fb2-9f9e-2d0f435b0fe3 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:15.061521 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac892589-d8c0-4fb2-9f9e-2d0f435b0fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:15.061521 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac892589-d8c0-4fb2-9f9e-2d0f435b0fe3 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:15.063366 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 620/1247] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:20:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:20:15.073451 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0b750-236b-4af3-ab18-a1ce08ae999a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:15.074141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0b750-236b-4af3-ab18-a1ce08ae999a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:15.074236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0b750-236b-4af3-ab18-a1ce08ae999a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:15.080970 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:15.098841 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c3f0b750-236b-4af3-ab18-a1ce08ae999a None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:20:15.100834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c3f0b750-236b-4af3-ab18-a1ce08ae999a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:20:15.199564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 628/1248] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:20:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:20:15.224772 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac892589-d8c0-4fb2-9f9e-2d0f435b0fe3 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:15.250073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:15.369694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac892589-d8c0-4fb2-9f9e-2d0f435b0fe3 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:15.413733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101448d-952a-4aa7-96c4-5473f8de4165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:15.414569 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101448d-952a-4aa7-96c4-5473f8de4165 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:15.414841 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101448d-952a-4aa7-96c4-5473f8de4165 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:15.415110 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101448d-952a-4aa7-96c4-5473f8de4165 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:15.427596 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101448d-952a-4aa7-96c4-5473f8de4165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:20:15.435800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-6101448d-952a-4aa7-96c4-5473f8de4165 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:20:15.435800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:15.435800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:15.435800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:15.435800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:15.435800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:15.435800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:15.435800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:15.435800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:15.435800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:15.435800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:15.435800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:15.435800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:20:15.435800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:20:15.435800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:20:15.435800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:20:15.436905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:20:15.436905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:20:15.436905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:20:15.436905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:20:15.436905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:15.436905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:15.436905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:15.436905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:15.436905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:15.436905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:15.436905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:15.436905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:20:15.436905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:15.436905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:15.436905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:15.436905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:15.437855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:15.437855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:15.437855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:15.437855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:15.437855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:20:15.437855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:20:15.437855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:20:15.437855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:20:15.437855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:20:15.437855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:20:15.437855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:20:15.437855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:20:15.437855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 621/1249] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:20:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:15.447439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0b750-236b-4af3-ab18-a1ce08ae999a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:15.589192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0b750-236b-4af3-ab18-a1ce08ae999a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:15.630548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:15.638741 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7cda6-0516-497c-b44f-294d38e30cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:15.639934 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7cda6-0516-497c-b44f-294d38e30cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:15.640433 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7cda6-0516-497c-b44f-294d38e30cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:15.640869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7cda6-0516-497c-b44f-294d38e30cb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:15.642088 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7cda6-0516-497c-b44f-294d38e30cb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:15.642938 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7cda6-0516-497c-b44f-294d38e30cb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:15.645451 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7cda6-0516-497c-b44f-294d38e30cb2 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:15.661135 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 629/1250] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:20:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:15.666800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101448d-952a-4aa7-96c4-5473f8de4165 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:15.779147 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:15.814740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101448d-952a-4aa7-96c4-5473f8de4165 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:15.866465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb153bc-5422-47ff-b35b-67b1c51aa6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:15.867475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb153bc-5422-47ff-b35b-67b1c51aa6cd None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:15.867922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb153bc-5422-47ff-b35b-67b1c51aa6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:15.868433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb153bc-5422-47ff-b35b-67b1c51aa6cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:15.884428 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb153bc-5422-47ff-b35b-67b1c51aa6cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:15.886013 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb153bc-5422-47ff-b35b-67b1c51aa6cd None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:15.906851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-edb153bc-5422-47ff-b35b-67b1c51aa6cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:15.906851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:15.906851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:15.906851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:15.906851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:15.906851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:15.906851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:15.906851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:15.906851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:15.906851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:15.906851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:15.906851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:15.906851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:20:15.906851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:20:15.906851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:20:15.907885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:20:15.907885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:15.907885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:15.907885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:15.907885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:15.907885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:15.907885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:15.907885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:15.907885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:20:15.907885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:15.907885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:15.907885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:15.907885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:15.907885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:15.907885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:15.907885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:15.907885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:15.910509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:20:15.910509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:20:15.910509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:15.910509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:20:15.910509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 622/1251] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:15.915760 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7cda6-0516-497c-b44f-294d38e30cb2 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:16.058127 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7cda6-0516-497c-b44f-294d38e30cb2 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:16.108479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4bc00-3454-40cf-9b31-400a3fc4218e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:16.109652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4bc00-3454-40cf-9b31-400a3fc4218e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:16.110136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4bc00-3454-40cf-9b31-400a3fc4218e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:16.110613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4bc00-3454-40cf-9b31-400a3fc4218e None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:16.112720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4bc00-3454-40cf-9b31-400a3fc4218e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:16.113272 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4bc00-3454-40cf-9b31-400a3fc4218e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:16.115881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4bc00-3454-40cf-9b31-400a3fc4218e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:16.132896 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 630/1252] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:16.139651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb153bc-5422-47ff-b35b-67b1c51aa6cd None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:16.283673 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb153bc-5422-47ff-b35b-67b1c51aa6cd None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:16.313961 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:16.325238 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010039f-ce5d-47d5-bfc2-48802ce140be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:16.326186 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010039f-ce5d-47d5-bfc2-48802ce140be None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:16.326469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010039f-ce5d-47d5-bfc2-48802ce140be None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:16.326774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010039f-ce5d-47d5-bfc2-48802ce140be None admin] PATH_INFO: `/v3/users/placement` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:16.336792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010039f-ce5d-47d5-bfc2-48802ce140be None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:16.338431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010039f-ce5d-47d5-bfc2-48802ce140be None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:16.356465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f010039f-ce5d-47d5-bfc2-48802ce140be None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 06:20:16.356465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:16.356465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:16.356465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:16.356465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:16.356465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:16.356465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:16.356465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:16.356465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:16.356465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:16.356465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:16.356465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:16.356465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:20:16.356465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:20:16.356465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:20:16.356465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:16.358303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:16.360145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:16.360145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:16.360145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:20:16.360145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:20:16.360145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:20:16.360145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:20:16.360145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:20:16.360145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:20:16.360145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 06:20:16.360145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:20:16.360145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 623/1253] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:16.364586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4bc00-3454-40cf-9b31-400a3fc4218e None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:16.442486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:16.507560 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4bc00-3454-40cf-9b31-400a3fc4218e None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:16.549507 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c88d53-d105-439c-a3fd-028d8bbe1fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:16.550203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c88d53-d105-439c-a3fd-028d8bbe1fef None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:16.550523 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c88d53-d105-439c-a3fd-028d8bbe1fef None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:16.550763 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c88d53-d105-439c-a3fd-028d8bbe1fef None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:16.551611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c88d53-d105-439c-a3fd-028d8bbe1fef None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:16.551920 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c88d53-d105-439c-a3fd-028d8bbe1fef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:16.553424 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c88d53-d105-439c-a3fd-028d8bbe1fef None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:16.572897 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 631/1254] 10.222.0.65 () {56 vars in 1143 bytes} [Thu May 19 06:20:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:20:16.579251 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010039f-ce5d-47d5-bfc2-48802ce140be None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:16.728281 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010039f-ce5d-47d5-bfc2-48802ce140be None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:16.770595 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238fb5a-c6ed-4cdf-bc7f-adc6acbd7622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:16.771652 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238fb5a-c6ed-4cdf-bc7f-adc6acbd7622 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:16.772141 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238fb5a-c6ed-4cdf-bc7f-adc6acbd7622 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:16.772635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238fb5a-c6ed-4cdf-bc7f-adc6acbd7622 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:16.853791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:16.870117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238fb5a-c6ed-4cdf-bc7f-adc6acbd7622 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:16.871987 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238fb5a-c6ed-4cdf-bc7f-adc6acbd7622 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:17.015486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 624/1255] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:20:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:20:17.021948 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c88d53-d105-439c-a3fd-028d8bbe1fef None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:17.178654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c88d53-d105-439c-a3fd-028d8bbe1fef None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:17.198490 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:17.225054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887ecd9-44d7-48b1-9433-bc1db1888e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:17.226203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887ecd9-44d7-48b1-9433-bc1db1888e97 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:17.226581 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887ecd9-44d7-48b1-9433-bc1db1888e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:17.226881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887ecd9-44d7-48b1-9433-bc1db1888e97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:17.242076 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887ecd9-44d7-48b1-9433-bc1db1888e97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:17.244275 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887ecd9-44d7-48b1-9433-bc1db1888e97 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:17.266516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-4887ecd9-44d7-48b1-9433-bc1db1888e97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:17.266516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:17.266516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:17.266516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:17.266516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:17.266516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:17.266516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:17.266516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:17.266516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:17.266516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:17.266516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:17.266516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:17.266516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:20:17.266516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:20:17.266516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:20:17.268421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:20:17.268421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:17.268421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:17.268421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:17.268421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:17.268421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:17.268421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:17.268421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:17.268421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:17.268421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:17.268421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:17.268421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:17.268421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:17.268421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:17.268421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:17.268421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:17.268421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:17.269550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:20:17.269550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:20:17.269550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:17.269550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:17.269550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 632/1256] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:17.279777 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238fb5a-c6ed-4cdf-bc7f-adc6acbd7622 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:17.416374 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:17.431985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238fb5a-c6ed-4cdf-bc7f-adc6acbd7622 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:17.484553 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f795359-a800-451a-b7a7-7ec970b5270d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:17.485731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f795359-a800-451a-b7a7-7ec970b5270d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:17.486225 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f795359-a800-451a-b7a7-7ec970b5270d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:17.486720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f795359-a800-451a-b7a7-7ec970b5270d None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:17.488303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f795359-a800-451a-b7a7-7ec970b5270d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:17.488862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f795359-a800-451a-b7a7-7ec970b5270d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:17.491536 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f795359-a800-451a-b7a7-7ec970b5270d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:17.516603 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 625/1257] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:20:17.518930 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887ecd9-44d7-48b1-9433-bc1db1888e97 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:17.667603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887ecd9-44d7-48b1-9433-bc1db1888e97 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:17.717433 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b3c1f-4b27-40f2-b25e-0bd93e65233d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:17.718495 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b3c1f-4b27-40f2-b25e-0bd93e65233d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:17.718915 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b3c1f-4b27-40f2-b25e-0bd93e65233d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:17.719314 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b3c1f-4b27-40f2-b25e-0bd93e65233d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:17.731590 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b3c1f-4b27-40f2-b25e-0bd93e65233d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:17.734282 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b3c1f-4b27-40f2-b25e-0bd93e65233d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:17.750207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-d34b3c1f-4b27-40f2-b25e-0bd93e65233d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:20:17.750207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:17.750207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:17.750207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:17.750207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:17.750207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:17.750207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:17.750207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:17.750207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:17.750207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:17.750207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:17.750207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:17.750207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:20:17.750207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:20:17.750207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:20:17.751512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:20:17.751512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:17.751512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:17.751512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:17.751512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:17.751512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:17.751512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:17.751512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:17.751512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:17.751512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:17.751512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:17.751512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:17.751512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:17.751512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:17.751512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:17.751512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:17.751512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:17.753183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:20:17.753183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:20:17.753183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:20:17.753183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:20:17.753183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:20:17.753183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:20:17.753183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:20:17.753183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:17.753183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 633/1258] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:20:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:17.758411 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f795359-a800-451a-b7a7-7ec970b5270d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:17.897617 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f795359-a800-451a-b7a7-7ec970b5270d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:17.956163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4d9fa-561c-4087-a3a8-a88641495370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:17.956978 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4d9fa-561c-4087-a3a8-a88641495370 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:17.957313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4d9fa-561c-4087-a3a8-a88641495370 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:17.957654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4d9fa-561c-4087-a3a8-a88641495370 None admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:17.958586 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e4d9fa-561c-4087-a3a8-a88641495370 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:17.958949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e4d9fa-561c-4087-a3a8-a88641495370 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:17.960569 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e4d9fa-561c-4087-a3a8-a88641495370 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:17.977369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 626/1259] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:20:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:17.984194 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b3c1f-4b27-40f2-b25e-0bd93e65233d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:18.016469 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:18.125558 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b3c1f-4b27-40f2-b25e-0bd93e65233d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:18.192406 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308778e-1903-4c30-b479-8dffd4db5fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:18.193062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308778e-1903-4c30-b479-8dffd4db5fae None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:18.193441 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308778e-1903-4c30-b479-8dffd4db5fae None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:18.193763 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308778e-1903-4c30-b479-8dffd4db5fae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:18.194675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308778e-1903-4c30-b479-8dffd4db5fae None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=1b76876d821e4bbbb8d5cad5bd26f559) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:18.194994 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308778e-1903-4c30-b479-8dffd4db5fae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:18.196472 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308778e-1903-4c30-b479-8dffd4db5fae None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:18.203454 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 634/1260] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:20:17 2022] GET /identity/v3/role_assignments?user.id=1b76876d821e4bbbb8d5cad5bd26f559&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:20.573187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0630bc5-1486-4e45-90f8-cca365b47e29 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:20.573802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0630bc5-1486-4e45-90f8-cca365b47e29 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:20.573802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0630bc5-1486-4e45-90f8-cca365b47e29 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:20.575246 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 627/1261] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:20:20.581969 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273490f-5675-48c1-82fd-1dbc1a024fff None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:20.582509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273490f-5675-48c1-82fd-1dbc1a024fff None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:20.582509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273490f-5675-48c1-82fd-1dbc1a024fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:20.601378 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f273490f-5675-48c1-82fd-1dbc1a024fff None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:20:20.602523 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f273490f-5675-48c1-82fd-1dbc1a024fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:20:20.696042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 635/1262] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:20:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:20:20.719495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0630bc5-1486-4e45-90f8-cca365b47e29 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:20.732988 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:20.845208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0630bc5-1486-4e45-90f8-cca365b47e29 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:20.883772 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b65df3-9ffd-4c3a-8b68-fe59a3c6e52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:20.884559 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b65df3-9ffd-4c3a-8b68-fe59a3c6e52f None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:20.884834 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b65df3-9ffd-4c3a-8b68-fe59a3c6e52f None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:20.885105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b65df3-9ffd-4c3a-8b68-fe59a3c6e52f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:20.897252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b65df3-9ffd-4c3a-8b68-fe59a3c6e52f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:20:20.905883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-02b65df3-9ffd-4c3a-8b68-fe59a3c6e52f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:20:20.905883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:20.905883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:20.905883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:20.905883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:20.905883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:20.905883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:20.905883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:20.905883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:20.905883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:20.905883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:20.905883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:20.905883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:20:20.905883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:20:20.905883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:20:20.905883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:20:20.911323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:20:20.911323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:20:20.911323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:20:20.911323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:20:20.911323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:20.911323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:20.911323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:20.911323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:20.911323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:20.911323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:20.911323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:20.911323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:20:20.911323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:20.911323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:20.911323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:20.911323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:20.912497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:20.912497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:20.912497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:20.912497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:20.912497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:20:20.912497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:20:20.912497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:20:20.912497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:20:20.912497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:20:20.912497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:20:20.912497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:20:20.912497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:20:20.912497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 628/1263] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:20:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:20.916067 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273490f-5675-48c1-82fd-1dbc1a024fff None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:20.943763 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:21.062160 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273490f-5675-48c1-82fd-1dbc1a024fff None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:21.113341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ed6af-0783-43ca-8167-1d9b0dd1ae4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:21.114205 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ed6af-0783-43ca-8167-1d9b0dd1ae4d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:21.114629 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ed6af-0783-43ca-8167-1d9b0dd1ae4d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:21.115126 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ed6af-0783-43ca-8167-1d9b0dd1ae4d None admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:21.116542 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ed6af-0783-43ca-8167-1d9b0dd1ae4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:21.117107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ed6af-0783-43ca-8167-1d9b0dd1ae4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:21.119759 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ed6af-0783-43ca-8167-1d9b0dd1ae4d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:21.136301 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 636/1264] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:20:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:21.142095 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b65df3-9ffd-4c3a-8b68-fe59a3c6e52f None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:21.213633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:21.252422 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b65df3-9ffd-4c3a-8b68-fe59a3c6e52f None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:21.296102 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ccdec-065f-4a77-b8e5-e685b9776ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:21.297211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ccdec-065f-4a77-b8e5-e685b9776ede None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:21.297741 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ccdec-065f-4a77-b8e5-e685b9776ede None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:21.298130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ccdec-065f-4a77-b8e5-e685b9776ede None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:21.311714 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ccdec-065f-4a77-b8e5-e685b9776ede None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:21.313171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ccdec-065f-4a77-b8e5-e685b9776ede None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:21.331753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-0f9ccdec-065f-4a77-b8e5-e685b9776ede None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:21.331753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:21.331753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:21.331753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:21.331753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:21.331753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:21.331753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:21.331753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:21.331753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:21.331753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:21.331753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:21.331753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:21.331753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:20:21.331753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:20:21.331753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:20:21.333066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:20:21.333066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:21.333066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:21.333066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:21.333066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:21.333066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:21.333066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:21.333066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:21.333066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:20:21.333066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:21.333066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:21.333066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:21.333066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:21.333066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:21.333066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:21.333066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:21.333066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:21.334692 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:20:21.334692 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:20:21.334692 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:21.334692 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:20:21.337459 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 629/1265] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:21.338858 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ed6af-0783-43ca-8167-1d9b0dd1ae4d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:21.427080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:21.450597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ed6af-0783-43ca-8167-1d9b0dd1ae4d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:21.485768 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59617f-7e43-4d6c-96c8-45c7e9d61b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:21.486976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59617f-7e43-4d6c-96c8-45c7e9d61b55 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:21.487257 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59617f-7e43-4d6c-96c8-45c7e9d61b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:21.487532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59617f-7e43-4d6c-96c8-45c7e9d61b55 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:21.488247 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59617f-7e43-4d6c-96c8-45c7e9d61b55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:21.488554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59617f-7e43-4d6c-96c8-45c7e9d61b55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:21.490002 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59617f-7e43-4d6c-96c8-45c7e9d61b55 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:21.505028 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 637/1266] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:20:21.512992 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ccdec-065f-4a77-b8e5-e685b9776ede None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:21.669369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ccdec-065f-4a77-b8e5-e685b9776ede None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:21.692252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:21.708136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255a0d6-9b65-403f-b322-71753fd849a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:21.708944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255a0d6-9b65-403f-b322-71753fd849a0 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:21.709228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255a0d6-9b65-403f-b322-71753fd849a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:21.709539 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255a0d6-9b65-403f-b322-71753fd849a0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:21.718331 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255a0d6-9b65-403f-b322-71753fd849a0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:21.719856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255a0d6-9b65-403f-b322-71753fd849a0 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:21.737488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f255a0d6-9b65-403f-b322-71753fd849a0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 06:20:21.737488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:21.737488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:21.737488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:21.737488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:21.737488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:21.737488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:21.737488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:21.737488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:21.737488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:21.737488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:21.737488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:21.737488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:20:21.737488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:20:21.737488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:20:21.737488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:21.739065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:21.740691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:21.740691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:21.740691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:20:21.740691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:20:21.740691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:20:21.740691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:20:21.740691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:20:21.740691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:20:21.740691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 06:20:21.740691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:20:21.740691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 630/1267] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:21.745243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59617f-7e43-4d6c-96c8-45c7e9d61b55 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:21.874783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59617f-7e43-4d6c-96c8-45c7e9d61b55 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:21.912532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3b7b1-f38e-43c5-b529-8dd977243767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:21.913231 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3b7b1-f38e-43c5-b529-8dd977243767 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:21.913539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3b7b1-f38e-43c5-b529-8dd977243767 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:21.913825 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3b7b1-f38e-43c5-b529-8dd977243767 None admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:21.914774 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3b7b1-f38e-43c5-b529-8dd977243767 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:21.915087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3b7b1-f38e-43c5-b529-8dd977243767 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:21.916535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3b7b1-f38e-43c5-b529-8dd977243767 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:21.926208 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:21.938961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 638/1268] 10.222.0.65 () {56 vars in 1143 bytes} [Thu May 19 06:20:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:20:21.944996 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255a0d6-9b65-403f-b322-71753fd849a0 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:22.090023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255a0d6-9b65-403f-b322-71753fd849a0 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:22.134979 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a8f11-7eb0-4d0a-b3ce-3562e4841c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:22.135790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a8f11-7eb0-4d0a-b3ce-3562e4841c7e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:22.136070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a8f11-7eb0-4d0a-b3ce-3562e4841c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:22.136343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a8f11-7eb0-4d0a-b3ce-3562e4841c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:22.149515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a8f11-7eb0-4d0a-b3ce-3562e4841c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:22.151086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a8f11-7eb0-4d0a-b3ce-3562e4841c7e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:22.170160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-972a8f11-7eb0-4d0a-b3ce-3562e4841c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:22.170160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:22.170160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:22.170160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:22.170160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:22.170160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:22.170160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:22.170160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:22.170160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:22.170160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:22.170160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:22.170160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:22.170160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:20:22.170160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:20:22.170160 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:20:22.171301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:20:22.171301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:22.171301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:22.171301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:22.171301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:22.171301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:22.171301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:22.171301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:22.171301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:20:22.171301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:22.171301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:22.171301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:22.171301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:22.171301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:22.171301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:22.171301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:22.171301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:22.172338 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:20:22.172338 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:20:22.172338 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:22.172338 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:20:22.172338 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 631/1269] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:22.177676 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3b7b1-f38e-43c5-b529-8dd977243767 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:22.302698 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3b7b1-f38e-43c5-b529-8dd977243767 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:22.342957 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898e943-acdc-4a1b-811e-d0a304eaeb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:22.343677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898e943-acdc-4a1b-811e-d0a304eaeb75 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:22.344052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898e943-acdc-4a1b-811e-d0a304eaeb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:22.344342 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898e943-acdc-4a1b-811e-d0a304eaeb75 None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:22.345094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898e943-acdc-4a1b-811e-d0a304eaeb75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:22.345490 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898e943-acdc-4a1b-811e-d0a304eaeb75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:22.347688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898e943-acdc-4a1b-811e-d0a304eaeb75 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:22.361086 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 639/1270] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:22.368515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a8f11-7eb0-4d0a-b3ce-3562e4841c7e None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:22.410270 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:22.504469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a8f11-7eb0-4d0a-b3ce-3562e4841c7e None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:22.556052 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0deae0-c077-4e6c-be35-154e06e3be1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:22.557009 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0deae0-c077-4e6c-be35-154e06e3be1a None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:22.557476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0deae0-c077-4e6c-be35-154e06e3be1a None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:22.557908 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0deae0-c077-4e6c-be35-154e06e3be1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:22.568271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0deae0-c077-4e6c-be35-154e06e3be1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:22.570975 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0deae0-c077-4e6c-be35-154e06e3be1a None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:22.593041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-cf0deae0-c077-4e6c-be35-154e06e3be1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:20:22.593041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:22.593041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:22.593041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:22.593041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:22.593041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:22.593041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:22.593041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:22.593041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:22.593041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:22.593041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:22.593041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:22.593041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:20:22.593041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:20:22.593041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:20:22.594220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:20:22.594220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:22.594220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:22.594220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:22.594220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:22.594220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:22.594220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:22.594220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:22.594220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:20:22.594220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:22.594220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:22.594220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:22.594220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:22.594220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:22.594220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:22.594220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:22.594220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:22.595340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:20:22.595340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:20:22.595340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:20:22.595340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:20:22.595340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:20:22.595340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:20:22.595340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:20:22.595340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:20:22.595340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 632/1271] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:20:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 06:20:22.602771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898e943-acdc-4a1b-811e-d0a304eaeb75 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:22.692230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:22.756480 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898e943-acdc-4a1b-811e-d0a304eaeb75 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:22.804846 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c92344-67da-4445-be9b-61d1ed94a03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:22.805545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c92344-67da-4445-be9b-61d1ed94a03e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:22.805847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c92344-67da-4445-be9b-61d1ed94a03e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:22.806194 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c92344-67da-4445-be9b-61d1ed94a03e None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:22.806911 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c92344-67da-4445-be9b-61d1ed94a03e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:22.807221 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c92344-67da-4445-be9b-61d1ed94a03e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:22.808678 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c92344-67da-4445-be9b-61d1ed94a03e None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:22.822607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 640/1272] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:20:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:22.828962 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0deae0-c077-4e6c-be35-154e06e3be1a None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:22.949989 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:22.961641 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0deae0-c077-4e6c-be35-154e06e3be1a None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:23.003499 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03681ed5-8dad-4867-bc3c-b1e5413bdb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:23.004467 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03681ed5-8dad-4867-bc3c-b1e5413bdb64 None admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:23.004750 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03681ed5-8dad-4867-bc3c-b1e5413bdb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:23.005027 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03681ed5-8dad-4867-bc3c-b1e5413bdb64 None admin] PATH_INFO: `/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/1b76876d821e4bbbb8d5cad5bd26f559/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:23.007384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03681ed5-8dad-4867-bc3c-b1e5413bdb64 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3d399dc95424b848b736c0d0298d9a9, user_id=1b76876d821e4bbbb8d5cad5bd26f559, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:23.009050 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03681ed5-8dad-4867-bc3c-b1e5413bdb64 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:23.024219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 633/1273] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 06:20:22 2022] PUT /identity/v3/projects/a3d399dc95424b848b736c0d0298d9a9/users/1b76876d821e4bbbb8d5cad5bd26f559/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:20:25.293208 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec4174-567d-43cb-8d8e-763b853c413c None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:25.293916 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec4174-567d-43cb-8d8e-763b853c413c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:25.293916 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec4174-567d-43cb-8d8e-763b853c413c None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:25.295238 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 641/1274] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:20:25.303025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeb3e8-1334-4c91-ac68-a2ac01bc4907 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:25.303576 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeb3e8-1334-4c91-ac68-a2ac01bc4907 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:25.303888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeb3e8-1334-4c91-ac68-a2ac01bc4907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:25.334702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-83eeb3e8-1334-4c91-ac68-a2ac01bc4907 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:20:25.335948 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-83eeb3e8-1334-4c91-ac68-a2ac01bc4907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:20:25.464551 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 634/1275] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:20:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:20:25.490463 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec4174-567d-43cb-8d8e-763b853c413c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:25.619887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec4174-567d-43cb-8d8e-763b853c413c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:25.641744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:25.663981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58b687-723c-4187-8280-f223b338cc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:25.664819 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58b687-723c-4187-8280-f223b338cc1e None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:25.665109 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58b687-723c-4187-8280-f223b338cc1e None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:25.665431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58b687-723c-4187-8280-f223b338cc1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:25.680062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df58b687-723c-4187-8280-f223b338cc1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 06:20:25.688014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-df58b687-723c-4187-8280-f223b338cc1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:20:25.688014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:25.688014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:25.688014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:25.688014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:25.688014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:25.688014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:25.688014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:25.688014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:25.688014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:25.688014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:25.688014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:25.688014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 06:20:25.688014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 06:20:25.688014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 06:20:25.688014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 06:20:25.689939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 06:20:25.689939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 06:20:25.689939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 06:20:25.689939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 06:20:25.689939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:25.689939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:25.689939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:25.689939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:25.689939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:25.689939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:25.689939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:25.689939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:25.689939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:25.689939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:25.689939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:25.689939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:25.691839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:25.691839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:25.691839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:25.691839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:25.691839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 06:20:25.691839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 06:20:25.691839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 06:20:25.691839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 06:20:25.691839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 06:20:25.691839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 06:20:25.691839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 06:20:25.691839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:25.691839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 642/1276] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 06:20:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:25.697269 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eeb3e8-1334-4c91-ac68-a2ac01bc4907 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:25.795434 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:25.817272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eeb3e8-1334-4c91-ac68-a2ac01bc4907 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:25.856631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b23c12-2f6f-42c6-830c-e04a960e97ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:25.857333 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b23c12-2f6f-42c6-830c-e04a960e97ee None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:25.857621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b23c12-2f6f-42c6-830c-e04a960e97ee None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:25.857900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b23c12-2f6f-42c6-830c-e04a960e97ee None admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:25.858679 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b23c12-2f6f-42c6-830c-e04a960e97ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:25.858989 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b23c12-2f6f-42c6-830c-e04a960e97ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:25.860401 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b23c12-2f6f-42c6-830c-e04a960e97ee None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:25.870385 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 635/1277] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 06:20:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:25.876327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58b687-723c-4187-8280-f223b338cc1e None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:25.987559 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58b687-723c-4187-8280-f223b338cc1e None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:26.024201 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108feb9a-2f30-4454-a519-14e7877f36ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:26.024997 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108feb9a-2f30-4454-a519-14e7877f36ff None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:26.025320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108feb9a-2f30-4454-a519-14e7877f36ff None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:26.025583 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108feb9a-2f30-4454-a519-14e7877f36ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:26.037392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108feb9a-2f30-4454-a519-14e7877f36ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:26.038922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108feb9a-2f30-4454-a519-14e7877f36ff None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:26.056714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-108feb9a-2f30-4454-a519-14e7877f36ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:26.056714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:26.056714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:26.056714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:26.056714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:26.056714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:26.056714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:26.056714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:26.056714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:26.056714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:26.056714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:26.056714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:26.056714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:20:26.056714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:20:26.056714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:20:26.057989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:20:26.057989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:26.057989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:26.057989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:26.057989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:26.057989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:26.057989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:26.057989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:26.057989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:26.057989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:26.057989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:26.057989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:26.057989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:26.057989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:26.057989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:26.057989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:26.057989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:26.059094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:20:26.059094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:20:26.059094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:26.059094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:26.059094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 643/1278] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:26.067469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b23c12-2f6f-42c6-830c-e04a960e97ee None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:26.189197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b23c12-2f6f-42c6-830c-e04a960e97ee None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:26.223637 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eaa40f-55fe-4179-a1a5-6a6faf83d95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:26.224298 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eaa40f-55fe-4179-a1a5-6a6faf83d95d None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:26.224583 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eaa40f-55fe-4179-a1a5-6a6faf83d95d None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:26.224864 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eaa40f-55fe-4179-a1a5-6a6faf83d95d None admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:26.225619 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eaa40f-55fe-4179-a1a5-6a6faf83d95d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:26.225927 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eaa40f-55fe-4179-a1a5-6a6faf83d95d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:26.227303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eaa40f-55fe-4179-a1a5-6a6faf83d95d None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:26.239256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 636/1279] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:26.245054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108feb9a-2f30-4454-a519-14e7877f36ff None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:26.277031 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:26.348765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108feb9a-2f30-4454-a519-14e7877f36ff None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:26.382732 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd18fb-2a03-4c9f-8e95-73ffa1147ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:26.383525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd18fb-2a03-4c9f-8e95-73ffa1147ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:26.383808 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd18fb-2a03-4c9f-8e95-73ffa1147ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:26.384085 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd18fb-2a03-4c9f-8e95-73ffa1147ae6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:26.391453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd18fb-2a03-4c9f-8e95-73ffa1147ae6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:26.393539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd18fb-2a03-4c9f-8e95-73ffa1147ae6 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:26.407595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-7ddd18fb-2a03-4c9f-8e95-73ffa1147ae6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 06:20:26.407595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:26.407595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:26.407595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:26.407595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:26.407595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:26.407595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:26.407595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:26.407595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:26.407595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:26.407595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:26.407595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:26.407595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 06:20:26.407595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 06:20:26.407595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 06:20:26.407595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:26.409822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:26.411023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:26.411023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:26.411023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 06:20:26.411023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 06:20:26.411023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 06:20:26.411023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 06:20:26.411023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 06:20:26.411023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 06:20:26.411023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 06:20:26.411023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:26.411023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 644/1280] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:26.415253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eaa40f-55fe-4179-a1a5-6a6faf83d95d None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:26.419226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:26.516860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eaa40f-55fe-4179-a1a5-6a6faf83d95d None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:26.557306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2c0b1-831c-4b10-a00a-7b705754c8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:26.558001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2c0b1-831c-4b10-a00a-7b705754c8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:26.558318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2c0b1-831c-4b10-a00a-7b705754c8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:26.558600 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2c0b1-831c-4b10-a00a-7b705754c8e2 None admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:26.559448 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2c0b1-831c-4b10-a00a-7b705754c8e2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:26.559763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2c0b1-831c-4b10-a00a-7b705754c8e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:26.561208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2c0b1-831c-4b10-a00a-7b705754c8e2 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:26.579453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 637/1281] 10.222.0.65 () {56 vars in 1143 bytes} [Thu May 19 06:20:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:26.585329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd18fb-2a03-4c9f-8e95-73ffa1147ae6 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:26.673595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:26.684142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd18fb-2a03-4c9f-8e95-73ffa1147ae6 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:26.718998 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b79e5-e5f1-4ef4-92da-0b30af3e417b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:26.719692 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b79e5-e5f1-4ef4-92da-0b30af3e417b None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:26.719982 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b79e5-e5f1-4ef4-92da-0b30af3e417b None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:26.720261 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b79e5-e5f1-4ef4-92da-0b30af3e417b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:26.789015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b79e5-e5f1-4ef4-92da-0b30af3e417b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:26.790426 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b79e5-e5f1-4ef4-92da-0b30af3e417b None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:26.877354 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:26.888075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 645/1282] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 06:20:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 06:20:26.894086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2c0b1-831c-4b10-a00a-7b705754c8e2 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:26.956461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:27.002340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2c0b1-831c-4b10-a00a-7b705754c8e2 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:27.036679 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65480a4-802b-4fe6-8a8b-d71a11a20a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:27.037498 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65480a4-802b-4fe6-8a8b-d71a11a20a68 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:27.037788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65480a4-802b-4fe6-8a8b-d71a11a20a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:27.038068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65480a4-802b-4fe6-8a8b-d71a11a20a68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:27.048244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65480a4-802b-4fe6-8a8b-d71a11a20a68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:27.050279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65480a4-802b-4fe6-8a8b-d71a11a20a68 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:27.068502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-f65480a4-802b-4fe6-8a8b-d71a11a20a68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:27.068502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:27.068502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:27.068502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:27.068502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:27.068502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:27.068502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:27.068502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:27.068502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:27.068502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:27.068502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:27.068502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:27.068502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 06:20:27.068502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 06:20:27.068502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 06:20:27.069806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 06:20:27.069806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:27.069806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:27.069806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:27.069806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:27.069806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:27.069806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:27.069806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:27.069806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:20:27.069806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:27.069806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:27.069806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:27.069806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:27.069806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:27.069806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:27.069806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:27.069806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:27.071253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 06:20:27.071253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 06:20:27.071253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 06:20:27.071253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:20:27.071253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 638/1283] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:27.075482 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b79e5-e5f1-4ef4-92da-0b30af3e417b None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:27.173813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b79e5-e5f1-4ef4-92da-0b30af3e417b None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:27.206972 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd725c-c66d-438a-bb9c-3a9e10ccd15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:27.207635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd725c-c66d-438a-bb9c-3a9e10ccd15d None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:27.207922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd725c-c66d-438a-bb9c-3a9e10ccd15d None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:27.208200 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd725c-c66d-438a-bb9c-3a9e10ccd15d None admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:27.208918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd725c-c66d-438a-bb9c-3a9e10ccd15d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:27.209227 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd725c-c66d-438a-bb9c-3a9e10ccd15d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:27.210631 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd725c-c66d-438a-bb9c-3a9e10ccd15d None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:27.222381 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 646/1284] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 06:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:27.227955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65480a4-802b-4fe6-8a8b-d71a11a20a68 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:27.327631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65480a4-802b-4fe6-8a8b-d71a11a20a68 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:27.341782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:27.362385 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a29686-b011-4399-8ef2-8963a282a317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:27.363185 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a29686-b011-4399-8ef2-8963a282a317 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:27.363496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a29686-b011-4399-8ef2-8963a282a317 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:27.363775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a29686-b011-4399-8ef2-8963a282a317 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:27.370396 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a29686-b011-4399-8ef2-8963a282a317 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:27.371839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a29686-b011-4399-8ef2-8963a282a317 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:27.386046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application [None req-85a29686-b011-4399-8ef2-8963a282a317 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:20:27.386046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:27.386046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:27.386046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:27.386046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:27.386046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:27.386046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:27.386046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:27.386046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:27.386046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:27.386046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:27.386046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:27.386046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 06:20:27.386046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 06:20:27.386046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 06:20:27.387118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 06:20:27.387118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:27.387118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:27.387118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:27.387118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:27.387118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:27.387118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:27.387118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:27.387118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application with Lock( May 19 06:20:27.387118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:27.387118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:27.387118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:27.387118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:27.387118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:27.387118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:27.387118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:27.387118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:27.388242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 06:20:27.388242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 06:20:27.388242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 06:20:27.388242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 06:20:27.388242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 06:20:27.388242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 06:20:27.388242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 06:20:27.388242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: ERROR keystone.server.flask.application May 19 06:20:27.388242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 639/1285] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 06:20:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:27.393509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd725c-c66d-438a-bb9c-3a9e10ccd15d None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:27.444605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:27.494615 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd725c-c66d-438a-bb9c-3a9e10ccd15d None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:27.528473 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a58983-79d9-43bc-8011-e97cd74c43a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:27.529138 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a58983-79d9-43bc-8011-e97cd74c43a9 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:27.529457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a58983-79d9-43bc-8011-e97cd74c43a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:27.529737 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a58983-79d9-43bc-8011-e97cd74c43a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:27.530521 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a58983-79d9-43bc-8011-e97cd74c43a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:27.530835 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a58983-79d9-43bc-8011-e97cd74c43a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:27.532324 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a58983-79d9-43bc-8011-e97cd74c43a9 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:27.545442 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 647/1286] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 06:20:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:27.552088 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a29686-b011-4399-8ef2-8963a282a317 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:27.651388 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a29686-b011-4399-8ef2-8963a282a317 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:27.685855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0dd8e-a43d-4be3-8963-8d91226826ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:27.686515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0dd8e-a43d-4be3-8963-8d91226826ee None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:27.686802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0dd8e-a43d-4be3-8963-8d91226826ee None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:27.687082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0dd8e-a43d-4be3-8963-8d91226826ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:27.687916 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0dd8e-a43d-4be3-8963-8d91226826ee None admin] RBAC: Adding query filter params (role.id=e0653fd3bcd64446979a2a9b403de431, scope.project.id=a3d399dc95424b848b736c0d0298d9a9, user.id=1b76876d821e4bbbb8d5cad5bd26f559) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:27.688233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0dd8e-a43d-4be3-8963-8d91226826ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:27.689785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0dd8e-a43d-4be3-8963-8d91226826ee None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:27.696679 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 640/1287] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 06:20:27 2022] GET /identity/v3/role_assignments?user.id=1b76876d821e4bbbb8d5cad5bd26f559&scope.project.id=a3d399dc95424b848b736c0d0298d9a9&role.id=e0653fd3bcd64446979a2a9b403de431 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:29.978245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda1e90-7623-4b4a-9686-3b60b8bd53b0 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:29.978819 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda1e90-7623-4b4a-9686-3b60b8bd53b0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:29.978923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda1e90-7623-4b4a-9686-3b60b8bd53b0 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:29.980493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 648/1288] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:20:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:20:29.988653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8ae6a-a2fa-49ae-826e-33da9bca2221 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:29.989256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8ae6a-a2fa-49ae-826e-33da9bca2221 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:29.989256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8ae6a-a2fa-49ae-826e-33da9bca2221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:30.013841 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-54b8ae6a-a2fa-49ae-826e-33da9bca2221 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:20:30.015082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-54b8ae6a-a2fa-49ae-826e-33da9bca2221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:20:30.052012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:30.093249 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 641/1289] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:20:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:20:30.117494 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda1e90-7623-4b4a-9686-3b60b8bd53b0 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:30.231469 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda1e90-7623-4b4a-9686-3b60b8bd53b0 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:30.244916 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:30.271441 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cb32e-15b8-4185-9058-0726029a3628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:30.272256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cb32e-15b8-4185-9058-0726029a3628 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:30.272545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cb32e-15b8-4185-9058-0726029a3628 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:30.272827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cb32e-15b8-4185-9058-0726029a3628 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:30.273530 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cb32e-15b8-4185-9058-0726029a3628 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:30.274971 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cb32e-15b8-4185-9058-0726029a3628 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:30.288607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-558cb32e-15b8-4185-9058-0726029a3628 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 06:20:30.288607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:30.288607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:20:30.288607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:20:30.288607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:20:30.288607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:20:30.288607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:20:30.288607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:20:30.288607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 06:20:30.288607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:30.288607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:20:30.288607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:30.288607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:30.288607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:30.288607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:30.288607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:30.288607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:30.289981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:30.289981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:30.289981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:30.289981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:30.289981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:30.289981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:30.289981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:20:30.289981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:20:30.289981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:20:30.289981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:20:30.289981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:30.289981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:30.289981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:30.289981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:30.289981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:30.289981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:30.290863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:30.290863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:30.290863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:30.290863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:30.290863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:30.290863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:30.290863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:30.290863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:30.290863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:30.290863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:30.290863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:20:30.290863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:20:30.290863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 06:20:30.290863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:30.290863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 649/1290] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 06:20:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:30.296267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8ae6a-a2fa-49ae-826e-33da9bca2221 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:30.403142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8ae6a-a2fa-49ae-826e-33da9bca2221 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:30.440732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b3f97-dff6-4c72-8c35-4c7d1ea15eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:30.441449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b3f97-dff6-4c72-8c35-4c7d1ea15eac None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:30.441738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b3f97-dff6-4c72-8c35-4c7d1ea15eac None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:30.442014 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b3f97-dff6-4c72-8c35-4c7d1ea15eac None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:30.442736 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b3f97-dff6-4c72-8c35-4c7d1ea15eac None admin] RBAC: Adding query filter params (name=placement) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:30.443043 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b3f97-dff6-4c72-8c35-4c7d1ea15eac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:30.444445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b3f97-dff6-4c72-8c35-4c7d1ea15eac None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:30.450052 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 642/1291] 10.222.0.65 () {56 vars in 1116 bytes} [Thu May 19 06:20:30 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:30.456923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cb32e-15b8-4185-9058-0726029a3628 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:30.562242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cb32e-15b8-4185-9058-0726029a3628 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:30.598209 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9a11d-f27c-4a1e-b425-117705eea341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:30.598876 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9a11d-f27c-4a1e-b425-117705eea341 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:30.599161 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9a11d-f27c-4a1e-b425-117705eea341 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:30.599436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9a11d-f27c-4a1e-b425-117705eea341 None admin] PATH_INFO: `/v3/services` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:30.600143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9a11d-f27c-4a1e-b425-117705eea341 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:30.600449 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9a11d-f27c-4a1e-b425-117705eea341 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:30.602210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9a11d-f27c-4a1e-b425-117705eea341 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:30.610343 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 650/1292] 10.222.0.65 () {56 vars in 1116 bytes} [Thu May 19 06:20:30 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:32.838805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7541b-480c-4baf-aba3-574b762cd24b None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:32.839356 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7541b-480c-4baf-aba3-574b762cd24b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:32.839356 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7541b-480c-4baf-aba3-574b762cd24b None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:32.840715 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 643/1293] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:20:32.846509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da56a7-3f5e-4d5d-ab23-e840fa1e9551 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:32.846931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da56a7-3f5e-4d5d-ab23-e840fa1e9551 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:32.847069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da56a7-3f5e-4d5d-ab23-e840fa1e9551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:32.864076 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-59da56a7-3f5e-4d5d-ab23-e840fa1e9551 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:20:32.865270 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-59da56a7-3f5e-4d5d-ab23-e840fa1e9551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:20:32.893975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:32.932298 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 651/1294] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:20:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:20:32.957436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7541b-480c-4baf-aba3-574b762cd24b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:32.992807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:33.070504 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7541b-480c-4baf-aba3-574b762cd24b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:33.105409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0ddb8-a84a-4e71-9e5f-d9d9888f2d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:33.105994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0ddb8-a84a-4e71-9e5f-d9d9888f2d27 None admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:33.106276 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0ddb8-a84a-4e71-9e5f-d9d9888f2d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:33.106550 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0ddb8-a84a-4e71-9e5f-d9d9888f2d27 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:33.107267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0ddb8-a84a-4e71-9e5f-d9d9888f2d27 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:33.108627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0ddb8-a84a-4e71-9e5f-d9d9888f2d27 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:33.122082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 644/1295] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 06:20:32 2022] POST /identity/v3/services => generated 246 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:20:35.419608 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d2711-003b-4783-82d5-bef368e08e1d None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:35.420163 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d2711-003b-4783-82d5-bef368e08e1d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:35.420581 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d2711-003b-4783-82d5-bef368e08e1d None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:35.422864 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 652/1296] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:20:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:20:35.430812 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f228e3c-26f0-4283-aef8-44f38696c651 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:35.431295 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f228e3c-26f0-4283-aef8-44f38696c651 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:35.431403 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f228e3c-26f0-4283-aef8-44f38696c651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:35.449103 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-1f228e3c-26f0-4283-aef8-44f38696c651 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:20:35.450333 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1f228e3c-26f0-4283-aef8-44f38696c651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:20:35.527215 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 645/1297] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:20:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:20:35.551394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d2711-003b-4783-82d5-bef368e08e1d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:35.672701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d2711-003b-4783-82d5-bef368e08e1d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:35.716062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3eba6-fc0a-4276-9bbb-61f600b336c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:35.716864 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3eba6-fc0a-4276-9bbb-61f600b336c5 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:35.717152 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3eba6-fc0a-4276-9bbb-61f600b336c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:35.717460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3eba6-fc0a-4276-9bbb-61f600b336c5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:35.718117 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd3eba6-fc0a-4276-9bbb-61f600b336c5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:35.719528 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd3eba6-fc0a-4276-9bbb-61f600b336c5 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:35.732131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-9cd3eba6-fc0a-4276-9bbb-61f600b336c5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 06:20:35.732131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:35.732131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:20:35.732131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:20:35.732131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:20:35.732131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:20:35.732131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:20:35.732131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:20:35.732131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 06:20:35.732131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:35.732131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:20:35.732131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:35.732131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:35.732131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:35.732131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:35.732131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:35.732131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:35.733415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:35.733415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:35.733415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:35.733415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:35.733415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:35.733415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:35.733415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:20:35.733415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:20:35.733415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:20:35.733415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:20:35.733415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:35.733415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:35.733415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:35.733415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:35.733415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:35.733415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:35.734318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:35.734318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:35.734318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:35.734318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:35.734318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:35.734318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:35.734318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:35.734318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:35.734318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:35.734318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:35.734318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:20:35.734318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:20:35.734318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 06:20:35.734318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:35.734318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 653/1298] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 06:20:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:35.740355 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f228e3c-26f0-4283-aef8-44f38696c651 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:35.759619 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:35.843022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f228e3c-26f0-4283-aef8-44f38696c651 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:35.877728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57552b6f-ed5e-4f31-a8b1-60233879a123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:35.878402 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57552b6f-ed5e-4f31-a8b1-60233879a123 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:35.878686 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57552b6f-ed5e-4f31-a8b1-60233879a123 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:35.878958 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57552b6f-ed5e-4f31-a8b1-60233879a123 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:35.879665 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57552b6f-ed5e-4f31-a8b1-60233879a123 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:35.879970 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57552b6f-ed5e-4f31-a8b1-60233879a123 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:35.881401 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57552b6f-ed5e-4f31-a8b1-60233879a123 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:35.886657 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 646/1299] 10.222.0.65 () {56 vars in 1116 bytes} [Thu May 19 06:20:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:35.892684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3eba6-fc0a-4276-9bbb-61f600b336c5 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:35.927252 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:35.997406 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3eba6-fc0a-4276-9bbb-61f600b336c5 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:36.031720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ff8dc-22f6-4fc0-8656-e8accb0ef306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:36.032382 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ff8dc-22f6-4fc0-8656-e8accb0ef306 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:36.032664 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ff8dc-22f6-4fc0-8656-e8accb0ef306 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:36.033010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ff8dc-22f6-4fc0-8656-e8accb0ef306 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:36.034503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ff8dc-22f6-4fc0-8656-e8accb0ef306 None admin] RBAC: Adding query filter params (interface=public, service_id=bb14eaad6e2e47ca9eb80d52135d4a1d, region_id=RegionOne) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:36.034852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ff8dc-22f6-4fc0-8656-e8accb0ef306 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:36.036241 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ff8dc-22f6-4fc0-8656-e8accb0ef306 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:36.042491 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 654/1300] 10.222.0.65 () {56 vars in 1251 bytes} [Thu May 19 06:20:35 2022] GET /identity/v3/endpoints?service_id=bb14eaad6e2e47ca9eb80d52135d4a1d&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:36.049108 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57552b6f-ed5e-4f31-a8b1-60233879a123 None admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:36.154695 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:36.173568 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57552b6f-ed5e-4f31-a8b1-60233879a123 None admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:36.208725 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b70ab5-d16b-4dd3-a138-349fbcb8d9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:36.209324 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b70ab5-d16b-4dd3-a138-349fbcb8d9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:36.209611 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b70ab5-d16b-4dd3-a138-349fbcb8d9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:36.209886 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b70ab5-d16b-4dd3-a138-349fbcb8d9c7 None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:36.210576 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b70ab5-d16b-4dd3-a138-349fbcb8d9c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:36.211940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b70ab5-d16b-4dd3-a138-349fbcb8d9c7 None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:36.218773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 647/1301] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 06:20:36 2022] GET /identity/v3/services => generated 2191 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:20:38.436881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e579d-14fc-4e52-a5de-6bea59b5a39b None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:38.437483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e579d-14fc-4e52-a5de-6bea59b5a39b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:38.437483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e579d-14fc-4e52-a5de-6bea59b5a39b None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:38.438837 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 655/1302] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:20:38.444841 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9bfa3-c083-4d3f-b4db-c19876dee3fe None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:38.445375 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9bfa3-c083-4d3f-b4db-c19876dee3fe None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:38.445606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9bfa3-c083-4d3f-b4db-c19876dee3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:38.464611 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-67e9bfa3-c083-4d3f-b4db-c19876dee3fe None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:20:38.465854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-67e9bfa3-c083-4d3f-b4db-c19876dee3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:20:38.535680 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 648/1303] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:20:38 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 06:20:38.561980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e579d-14fc-4e52-a5de-6bea59b5a39b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:38.668387 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:38.676468 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e579d-14fc-4e52-a5de-6bea59b5a39b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:38.712995 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccc827-0e6a-4ec8-87c0-12ff9b0ed123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:38.714016 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccc827-0e6a-4ec8-87c0-12ff9b0ed123 None admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:38.714306 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccc827-0e6a-4ec8-87c0-12ff9b0ed123 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:38.714583 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccc827-0e6a-4ec8-87c0-12ff9b0ed123 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:38.715238 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ccc827-0e6a-4ec8-87c0-12ff9b0ed123 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:38.716617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ccc827-0e6a-4ec8-87c0-12ff9b0ed123 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:38.729646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application [None req-40ccc827-0e6a-4ec8-87c0-12ff9b0ed123 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 06:20:38.729646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:38.729646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 06:20:38.729646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 06:20:38.729646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 06:20:38.729646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 06:20:38.729646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 06:20:38.729646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:20:38.729646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 06:20:38.729646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:38.729646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 06:20:38.729646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:38.729646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 06:20:38.729646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 06:20:38.729646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 06:20:38.729646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 06:20:38.729646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 06:20:38.731607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 06:20:38.731607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 06:20:38.731607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 06:20:38.731607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 06:20:38.731607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 06:20:38.731607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 06:20:38.731607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 06:20:38.731607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 06:20:38.731607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 06:20:38.731607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 06:20:38.731607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 06:20:38.731607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 06:20:38.731607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 06:20:38.731607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 06:20:38.731607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 06:20:38.731607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.get_or_create( May 19 06:20:38.733681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 06:20:38.733681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application with Lock( May 19 06:20:38.733681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 06:20:38.733681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self._enter() May 19 06:20:38.733681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 06:20:38.733681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 06:20:38.733681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 06:20:38.733681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application return self.creator() May 19 06:20:38.733681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 06:20:38.733681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application created_value = creator( May 19 06:20:38.733681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 06:20:38.733681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 06:20:38.733681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 06:20:38.733681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: ERROR keystone.server.flask.application May 19 06:20:38.733681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 656/1304] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 06:20:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 06:20:38.739705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9bfa3-c083-4d3f-b4db-c19876dee3fe None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:38.835953 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:38.859029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9bfa3-c083-4d3f-b4db-c19876dee3fe None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:38.894600 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f27a0f-9932-4295-88ac-779e50cd432e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:38.895292 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f27a0f-9932-4295-88ac-779e50cd432e None admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:38.895578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f27a0f-9932-4295-88ac-779e50cd432e None admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:38.895860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f27a0f-9932-4295-88ac-779e50cd432e None admin] PATH_INFO: `/v3/services` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:38.896600 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f27a0f-9932-4295-88ac-779e50cd432e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:20:38.896911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f27a0f-9932-4295-88ac-779e50cd432e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:38.898352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f27a0f-9932-4295-88ac-779e50cd432e None admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:38.904279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 649/1305] 10.222.0.65 () {56 vars in 1116 bytes} [Thu May 19 06:20:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:20:38.910351 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccc827-0e6a-4ec8-87c0-12ff9b0ed123 None admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:20:39.014628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccc827-0e6a-4ec8-87c0-12ff9b0ed123 None admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:20:39.051718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6597f-a0a3-483b-a92d-7e4d05715353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:20:39.052386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6597f-a0a3-483b-a92d-7e4d05715353 None admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:20:39.052674 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6597f-a0a3-483b-a92d-7e4d05715353 None admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:20:39.052954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6597f-a0a3-483b-a92d-7e4d05715353 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:20:39.053748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6597f-a0a3-483b-a92d-7e4d05715353 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:20:39.055155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6597f-a0a3-483b-a92d-7e4d05715353 None admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:20:39.076188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 657/1306] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 06:20:38 2022] POST /identity/v3/endpoints => generated 331 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:20:48.900750 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=30 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:20:49.069993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:21:46.751328 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc9bfa-1d9b-4dc4-bb7b-f5fce55e75a4 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:21:46.751867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc9bfa-1d9b-4dc4-bb7b-f5fce55e75a4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:21:46.751867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc9bfa-1d9b-4dc4-bb7b-f5fce55e75a4 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:21:46.758762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62895284-cc42-4eac-93a7-8990201bc264 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:21:46.759319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62895284-cc42-4eac-93a7-8990201bc264 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:21:46.759319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62895284-cc42-4eac-93a7-8990201bc264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:21:46.762755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 650/1307] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:21:46 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:21:46.844847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-62895284-cc42-4eac-93a7-8990201bc264 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:21:46.846117 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-62895284-cc42-4eac-93a7-8990201bc264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:21:46.987202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 658/1308] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:21:47.039838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c35f0c-fe67-4000-acc9-a6d35363b1d8 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:21:47.040284 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c35f0c-fe67-4000-acc9-a6d35363b1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:21:47.040652 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c35f0c-fe67-4000-acc9-a6d35363b1d8 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:21:47.042461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 651/1309] 10.222.0.65 () {54 vars in 964 bytes} [Thu May 19 06:21:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:21:47.049482 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b67d8-61d3-4d08-9c65-7a5eb1e81cd4 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:21:47.050114 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b67d8-61d3-4d08-9c65-7a5eb1e81cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:21:47.050551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b67d8-61d3-4d08-9c65-7a5eb1e81cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:21:47.147089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-ad8b67d8-61d3-4d08-9c65-7a5eb1e81cd4 None None] MFA Rules not processed for user `226f4700a516415cbbd272fdfad7af58`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:21:47.148655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ad8b67d8-61d3-4d08-9c65-7a5eb1e81cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:21:47.175023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:21:47.264514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 659/1310] 10.222.0.65 () {58 vars in 1061 bytes} [Thu May 19 06:21:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:21:47.279273 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c35f0c-fe67-4000-acc9-a6d35363b1d8 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:21:47.279273 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b9c35f0c-fe67-4000-acc9-a6d35363b1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:21:47.480396 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c35f0c-fe67-4000-acc9-a6d35363b1d8 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:21:47.587967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919ba1e-f992-4156-886c-918a9888955f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:21:47.589168 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919ba1e-f992-4156-886c-918a9888955f service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:21:47.589737 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919ba1e-f992-4156-886c-918a9888955f service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:21:47.590306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919ba1e-f992-4156-886c-918a9888955f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:21:47.592601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b919ba1e-f992-4156-886c-918a9888955f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:21:47.685080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:21:47.765081 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919ba1e-f992-4156-886c-918a9888955f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:21:47.766361 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919ba1e-f992-4156-886c-918a9888955f service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:21:47.799105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 652/1311] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:21:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:21:50.438775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f26d8-2e97-44bc-acee-016bd0bbfe37 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:21:50.439347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f26d8-2e97-44bc-acee-016bd0bbfe37 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:21:50.440285 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f26d8-2e97-44bc-acee-016bd0bbfe37 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:21:50.442407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 660/1312] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:21:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:21:50.448293 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e082eb-f4ef-4f90-b9c8-a99772bc699e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:21:50.448809 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e082eb-f4ef-4f90-b9c8-a99772bc699e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:21:50.448893 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e082eb-f4ef-4f90-b9c8-a99772bc699e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:21:50.535910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-52e082eb-f4ef-4f90-b9c8-a99772bc699e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:21:50.537116 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-52e082eb-f4ef-4f90-b9c8-a99772bc699e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:21:50.655678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 653/1313] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:21:50.696764 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f26d8-2e97-44bc-acee-016bd0bbfe37 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:21:50.698389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cf1f26d8-2e97-44bc-acee-016bd0bbfe37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:21:50.836478 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f26d8-2e97-44bc-acee-016bd0bbfe37 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:21:50.923839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:21:50.928138 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e64d26-f935-4b74-9660-3e2e013e2218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:21:50.928598 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e64d26-f935-4b74-9660-3e2e013e2218 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:21:50.929225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e64d26-f935-4b74-9660-3e2e013e2218 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:21:50.930128 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e64d26-f935-4b74-9660-3e2e013e2218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:21:50.931970 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-22e64d26-f935-4b74-9660-3e2e013e2218 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:21:51.064190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e64d26-f935-4b74-9660-3e2e013e2218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:21:51.065763 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e64d26-f935-4b74-9660-3e2e013e2218 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:21:51.087701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 661/1314] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:21:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:21:53.789787 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21533df8-6f4e-421d-95bf-39d27fcf716d None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:21:53.790366 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21533df8-6f4e-421d-95bf-39d27fcf716d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:21:53.790635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21533df8-6f4e-421d-95bf-39d27fcf716d None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:21:53.792833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 654/1315] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:21:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:21:53.798454 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57768d-d70e-4157-a828-077194524855 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:21:53.798979 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57768d-d70e-4157-a828-077194524855 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:21:53.799779 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57768d-d70e-4157-a828-077194524855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:21:53.868229 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-8c57768d-d70e-4157-a828-077194524855 None None] MFA Rules not processed for user `b8be4e8c78444c25a8bd96f5851b92a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:21:53.869494 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8c57768d-d70e-4157-a828-077194524855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:21:53.923784 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:21:53.972682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 662/1316] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:21:53 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:21:54.013462 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21533df8-6f4e-421d-95bf-39d27fcf716d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:21:54.015022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-21533df8-6f4e-421d-95bf-39d27fcf716d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:21:54.025493 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:21:54.133157 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21533df8-6f4e-421d-95bf-39d27fcf716d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:21:54.212737 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1363e72-df2a-48cd-b0f5-33a05c723ade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:21:54.213421 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1363e72-df2a-48cd-b0f5-33a05c723ade service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:21:54.213708 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1363e72-df2a-48cd-b0f5-33a05c723ade service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:21:54.213983 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1363e72-df2a-48cd-b0f5-33a05c723ade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:21:54.215676 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b1363e72-df2a-48cd-b0f5-33a05c723ade service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:21:54.281260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:21:54.334368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1363e72-df2a-48cd-b0f5-33a05c723ade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:21:54.335861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1363e72-df2a-48cd-b0f5-33a05c723ade service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:21:54.355256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 655/1317] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:21:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:21:57.363909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5d2cc-8129-442f-bec9-7ee96bf7b88c None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:21:57.364609 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5d2cc-8129-442f-bec9-7ee96bf7b88c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:21:57.364609 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5d2cc-8129-442f-bec9-7ee96bf7b88c None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:21:57.365920 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 663/1318] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:21:57.371694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bba1c9-b80e-47c4-872d-6274d917dd22 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:21:57.372135 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bba1c9-b80e-47c4-872d-6274d917dd22 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:21:57.372258 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bba1c9-b80e-47c4-872d-6274d917dd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:21:57.441691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a0bba1c9-b80e-47c4-872d-6274d917dd22 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:21:57.442911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a0bba1c9-b80e-47c4-872d-6274d917dd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:21:57.542761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 656/1319] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:21:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:21:57.603371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e6517-3741-4105-b2f8-72e4d1f43a69 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:21:57.603583 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e6517-3741-4105-b2f8-72e4d1f43a69 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:21:57.603844 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e6517-3741-4105-b2f8-72e4d1f43a69 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:21:57.605215 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 664/1320] 10.222.0.65 () {54 vars in 964 bytes} [Thu May 19 06:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:21:57.611407 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96ed62-403c-4eb4-aaa4-f246bb2b3611 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:21:57.611737 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96ed62-403c-4eb4-aaa4-f246bb2b3611 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:21:57.612023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96ed62-403c-4eb4-aaa4-f246bb2b3611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:21:57.632944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:21:57.698919 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-aa96ed62-403c-4eb4-aaa4-f246bb2b3611 None None] MFA Rules not processed for user `226f4700a516415cbbd272fdfad7af58`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:21:57.700175 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-aa96ed62-403c-4eb4-aaa4-f246bb2b3611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:21:57.794606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 657/1321] 10.222.0.65 () {58 vars in 1061 bytes} [Thu May 19 06:21:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:21:57.804234 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e6517-3741-4105-b2f8-72e4d1f43a69 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:21:57.805870 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-944e6517-3741-4105-b2f8-72e4d1f43a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:21:57.925803 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e6517-3741-4105-b2f8-72e4d1f43a69 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:21:58.002080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f9719-05f8-48d1-bd3b-6936eaa17f48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:21:58.003288 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f9719-05f8-48d1-bd3b-6936eaa17f48 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:21:58.003569 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f9719-05f8-48d1-bd3b-6936eaa17f48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:21:58.003843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f9719-05f8-48d1-bd3b-6936eaa17f48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:21:58.005539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-048f9719-05f8-48d1-bd3b-6936eaa17f48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:21:58.012624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:21:58.115465 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f9719-05f8-48d1-bd3b-6936eaa17f48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:21:58.116918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f9719-05f8-48d1-bd3b-6936eaa17f48 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:21:58.134855 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 665/1322] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:21:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:22:01.607868 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafef67-d690-4bb0-82e7-38d096826465 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:01.608433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafef67-d690-4bb0-82e7-38d096826465 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:01.608433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafef67-d690-4bb0-82e7-38d096826465 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:01.609911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 658/1323] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:22:01.616276 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36d5c1-20ee-4462-980c-6ea3fb279c08 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:01.616804 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36d5c1-20ee-4462-980c-6ea3fb279c08 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:01.616903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36d5c1-20ee-4462-980c-6ea3fb279c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:01.694058 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3c36d5c1-20ee-4462-980c-6ea3fb279c08 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:01.696243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3c36d5c1-20ee-4462-980c-6ea3fb279c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:01.794554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:22:01.839167 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 666/1324] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:22:01.875420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafef67-d690-4bb0-82e7-38d096826465 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:22:01.877035 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2dafef67-d690-4bb0-82e7-38d096826465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:01.995471 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafef67-d690-4bb0-82e7-38d096826465 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:22:02.009445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:22:02.077744 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94788f1c-951c-4dc6-8a3a-53b73f77b0da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:22:02.078398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94788f1c-951c-4dc6-8a3a-53b73f77b0da service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:02.078683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94788f1c-951c-4dc6-8a3a-53b73f77b0da service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:02.078961 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94788f1c-951c-4dc6-8a3a-53b73f77b0da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:02.081418 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-94788f1c-951c-4dc6-8a3a-53b73f77b0da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:02.200122 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94788f1c-951c-4dc6-8a3a-53b73f77b0da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:22:02.201557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94788f1c-951c-4dc6-8a3a-53b73f77b0da service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:22:02.221857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 659/1325] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:22:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:22:04.602744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c60b2c-1aed-4d2d-bd16-2b4c6b557c42 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:04.603302 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c60b2c-1aed-4d2d-bd16-2b4c6b557c42 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:04.603600 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c60b2c-1aed-4d2d-bd16-2b4c6b557c42 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:04.605993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 667/1326] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:22:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:22:04.614016 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693ae59-3d3e-4614-b56e-d65501b86ae7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:04.614788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693ae59-3d3e-4614-b56e-d65501b86ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:04.614788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693ae59-3d3e-4614-b56e-d65501b86ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:04.675693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:22:04.698439 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5693ae59-3d3e-4614-b56e-d65501b86ae7 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:04.699659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5693ae59-3d3e-4614-b56e-d65501b86ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:04.804627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 660/1327] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:22:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:22:04.841271 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c60b2c-1aed-4d2d-bd16-2b4c6b557c42 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:22:04.842838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d8c60b2c-1aed-4d2d-bd16-2b4c6b557c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:04.961587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c60b2c-1aed-4d2d-bd16-2b4c6b557c42 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:22:05.050232 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d3359-2b91-46b8-84f5-c574d7dd0f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:22:05.050891 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d3359-2b91-46b8-84f5-c574d7dd0f6d service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:05.051178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d3359-2b91-46b8-84f5-c574d7dd0f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:05.051454 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d3359-2b91-46b8-84f5-c574d7dd0f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:05.053137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-760d3359-2b91-46b8-84f5-c574d7dd0f6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:05.078248 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:22:05.195552 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d3359-2b91-46b8-84f5-c574d7dd0f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:22:05.197054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d3359-2b91-46b8-84f5-c574d7dd0f6d service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:22:05.215961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 668/1328] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:22:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:22:07.675169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fa297-5781-4e84-babe-db40a85d2363 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:07.675911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fa297-5781-4e84-babe-db40a85d2363 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:07.675911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fa297-5781-4e84-babe-db40a85d2363 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:07.677132 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 661/1329] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:22:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:22:07.683094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092c46f-28a2-4412-acad-c4f5050b5796 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:07.683633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092c46f-28a2-4412-acad-c4f5050b5796 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:07.683731 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092c46f-28a2-4412-acad-c4f5050b5796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:07.758514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f092c46f-28a2-4412-acad-c4f5050b5796 None None] MFA Rules not processed for user `b8be4e8c78444c25a8bd96f5851b92a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:07.759769 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f092c46f-28a2-4412-acad-c4f5050b5796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:07.839737 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:22:07.878529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 669/1330] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:22:07 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:22:07.916702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fa297-5781-4e84-babe-db40a85d2363 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:22:07.918445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f46fa297-5781-4e84-babe-db40a85d2363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:08.046845 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fa297-5781-4e84-babe-db40a85d2363 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:22:08.069945 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:22:08.131069 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba7a05-c8b9-4e97-a358-bc055dd0f279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:22:08.131733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba7a05-c8b9-4e97-a358-bc055dd0f279 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:08.132019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba7a05-c8b9-4e97-a358-bc055dd0f279 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:08.132299 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba7a05-c8b9-4e97-a358-bc055dd0f279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:08.134079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cdba7a05-c8b9-4e97-a358-bc055dd0f279 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:08.262481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba7a05-c8b9-4e97-a358-bc055dd0f279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:22:08.264161 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba7a05-c8b9-4e97-a358-bc055dd0f279 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:22:08.288024 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 662/1331] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:22:07 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:22:10.930607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1394d5-9df2-4e30-9577-620a95db4320 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:10.931244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1394d5-9df2-4e30-9577-620a95db4320 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:10.931244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1394d5-9df2-4e30-9577-620a95db4320 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:10.932565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 670/1332] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:22:10.938233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a4ae8-e16b-406e-ad75-d4f88e49c015 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:10.938716 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a4ae8-e16b-406e-ad75-d4f88e49c015 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:10.938843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a4ae8-e16b-406e-ad75-d4f88e49c015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:11.008728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:22:11.012345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b35a4ae8-e16b-406e-ad75-d4f88e49c015 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:11.014111 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b35a4ae8-e16b-406e-ad75-d4f88e49c015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:11.133399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 663/1333] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:22:11.171065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1394d5-9df2-4e30-9577-620a95db4320 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:22:11.172731 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6d1394d5-9df2-4e30-9577-620a95db4320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:11.316954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1394d5-9df2-4e30-9577-620a95db4320 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:22:11.418233 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee29220-afe8-4cf4-ad2c-24bc70e7d652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:22:11.418802 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee29220-afe8-4cf4-ad2c-24bc70e7d652 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:11.419084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee29220-afe8-4cf4-ad2c-24bc70e7d652 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:11.419360 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee29220-afe8-4cf4-ad2c-24bc70e7d652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:11.421065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fee29220-afe8-4cf4-ad2c-24bc70e7d652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:11.454112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:22:11.580418 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee29220-afe8-4cf4-ad2c-24bc70e7d652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:22:11.583066 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee29220-afe8-4cf4-ad2c-24bc70e7d652 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:22:11.619298 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 671/1334] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:22:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:22:14.125415 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f003139-553f-46c5-8d1b-2c61c094cdac None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:14.125981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f003139-553f-46c5-8d1b-2c61c094cdac None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:14.125981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f003139-553f-46c5-8d1b-2c61c094cdac None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:14.128411 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 664/1335] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:22:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:22:14.134232 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178e46f-86b6-4608-97d7-6c2bf9f44e67 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:14.134731 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178e46f-86b6-4608-97d7-6c2bf9f44e67 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:14.134886 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178e46f-86b6-4608-97d7-6c2bf9f44e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:14.212517 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a178e46f-86b6-4608-97d7-6c2bf9f44e67 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:14.213840 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a178e46f-86b6-4608-97d7-6c2bf9f44e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:14.303448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:22:14.329156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 672/1336] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:22:14.373862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f003139-553f-46c5-8d1b-2c61c094cdac None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:22:14.375412 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7f003139-553f-46c5-8d1b-2c61c094cdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:14.515224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f003139-553f-46c5-8d1b-2c61c094cdac None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:22:14.554763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:22:14.607807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a780611-a255-466b-aea6-5f8622d6dde9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:22:14.608472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a780611-a255-466b-aea6-5f8622d6dde9 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:14.608761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a780611-a255-466b-aea6-5f8622d6dde9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:14.609039 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a780611-a255-466b-aea6-5f8622d6dde9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:14.610783 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9a780611-a255-466b-aea6-5f8622d6dde9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:14.743838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a780611-a255-466b-aea6-5f8622d6dde9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:22:14.745368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a780611-a255-466b-aea6-5f8622d6dde9 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:22:14.766417 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 665/1337] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:22:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:22:17.513508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862002d-9ffa-4a39-94f1-ef25d4315a1e None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:17.514038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862002d-9ffa-4a39-94f1-ef25d4315a1e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:17.514158 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862002d-9ffa-4a39-94f1-ef25d4315a1e None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:17.515469 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 673/1338] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:22:17.521679 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465d8e5-a2ab-4048-bd0b-de19c5d3e042 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:17.522132 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465d8e5-a2ab-4048-bd0b-de19c5d3e042 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:17.522249 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465d8e5-a2ab-4048-bd0b-de19c5d3e042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:17.597483 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-4465d8e5-a2ab-4048-bd0b-de19c5d3e042 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:17.598631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4465d8e5-a2ab-4048-bd0b-de19c5d3e042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:17.607533 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:22:17.708675 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 666/1339] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:22:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:22:17.744335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862002d-9ffa-4a39-94f1-ef25d4315a1e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:22:17.746517 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3862002d-9ffa-4a39-94f1-ef25d4315a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:17.842755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862002d-9ffa-4a39-94f1-ef25d4315a1e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:22:17.846426 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6c87c-6376-4f12-8ab7-c5b45e298420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:22:17.847067 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6c87c-6376-4f12-8ab7-c5b45e298420 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:17.847352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6c87c-6376-4f12-8ab7-c5b45e298420 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:17.847690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6c87c-6376-4f12-8ab7-c5b45e298420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:17.856951 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6c87c-6376-4f12-8ab7-c5b45e298420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:22:17.858433 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6c87c-6376-4f12-8ab7-c5b45e298420 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:22:17.868491 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 674/1340] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:22:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:22:22.996272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f0780-c943-45a1-9522-e3ce4504cf48 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:22.997072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f0780-c943-45a1-9522-e3ce4504cf48 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:22.997256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f0780-c943-45a1-9522-e3ce4504cf48 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:22.998730 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 667/1341] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:22:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:22:23.004926 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc689d-5591-41b0-9045-aa028ffbf70d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:23.005462 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc689d-5591-41b0-9045-aa028ffbf70d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:23.005617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc689d-5591-41b0-9045-aa028ffbf70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:23.038412 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-ddcc689d-5591-41b0-9045-aa028ffbf70d None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:23.039775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ddcc689d-5591-41b0-9045-aa028ffbf70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:23.130716 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 675/1342] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:22:23.198993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f0780-c943-45a1-9522-e3ce4504cf48 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:22:23.210847 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f0780-c943-45a1-9522-e3ce4504cf48 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:22:23.214547 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067912f9-9143-4c4e-a1cf-c579b1ca358d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:22:23.215201 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067912f9-9143-4c4e-a1cf-c579b1ca358d service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:23.215483 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067912f9-9143-4c4e-a1cf-c579b1ca358d service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:23.215761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067912f9-9143-4c4e-a1cf-c579b1ca358d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:23.226636 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067912f9-9143-4c4e-a1cf-c579b1ca358d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:22:23.228315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067912f9-9143-4c4e-a1cf-c579b1ca358d service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:22:23.240233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 668/1343] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:22:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:22:26.353050 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b0ac0-997e-4831-afc1-8d3559923748 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:26.353618 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b0ac0-997e-4831-afc1-8d3559923748 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:26.353918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b0ac0-997e-4831-afc1-8d3559923748 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:26.356041 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 676/1344] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:22:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:22:26.363072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801178f3-6b38-4492-b4d7-d4c159e93666 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:26.363615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801178f3-6b38-4492-b4d7-d4c159e93666 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:26.363843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801178f3-6b38-4492-b4d7-d4c159e93666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:26.383381 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-801178f3-6b38-4492-b4d7-d4c159e93666 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:26.384593 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-801178f3-6b38-4492-b4d7-d4c159e93666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:26.393652 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 669/1345] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:22:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:22:26.431174 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b0ac0-997e-4831-afc1-8d3559923748 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:22:26.441967 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b0ac0-997e-4831-afc1-8d3559923748 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:22:26.445706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26e64d-5525-434a-b918-d94686f794c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:22:26.446352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26e64d-5525-434a-b918-d94686f794c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:26.446693 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26e64d-5525-434a-b918-d94686f794c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:26.446971 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26e64d-5525-434a-b918-d94686f794c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:26.457721 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d26e64d-5525-434a-b918-d94686f794c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:22:26.459286 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d26e64d-5525-434a-b918-d94686f794c3 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:22:26.470695 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 677/1346] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:22:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:22:31.164732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476aaf3-a074-4ed9-a6ee-2ccf8572557f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:31.165680 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476aaf3-a074-4ed9-a6ee-2ccf8572557f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:31.165680 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476aaf3-a074-4ed9-a6ee-2ccf8572557f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:31.168063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 670/1347] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:22:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:22:31.172396 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544776e2-7898-4202-a0fe-fc76d7916f0e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:31.173259 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544776e2-7898-4202-a0fe-fc76d7916f0e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:31.173259 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544776e2-7898-4202-a0fe-fc76d7916f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:31.195863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-544776e2-7898-4202-a0fe-fc76d7916f0e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:31.197621 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-544776e2-7898-4202-a0fe-fc76d7916f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:31.208981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 678/1348] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:22:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:22:31.248136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476aaf3-a074-4ed9-a6ee-2ccf8572557f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:22:31.251108 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d476aaf3-a074-4ed9-a6ee-2ccf8572557f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:31.274508 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476aaf3-a074-4ed9-a6ee-2ccf8572557f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:22:31.280896 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569445bf-e158-4157-b2eb-f070796b552a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:22:31.282142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569445bf-e158-4157-b2eb-f070796b552a service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:31.282734 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569445bf-e158-4157-b2eb-f070796b552a service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:31.283931 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569445bf-e158-4157-b2eb-f070796b552a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:31.301106 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569445bf-e158-4157-b2eb-f070796b552a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:22:31.304013 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569445bf-e158-4157-b2eb-f070796b552a service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:22:31.322461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 671/1349] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:22:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:22:37.478329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546ba29-fe36-41d5-9a10-fb0e2ddade78 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:37.478829 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546ba29-fe36-41d5-9a10-fb0e2ddade78 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:37.479251 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546ba29-fe36-41d5-9a10-fb0e2ddade78 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:37.481583 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 679/1350] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 06:22:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:22:37.501341 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e7a48-2843-4cf4-934d-53aabcc33d9e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:37.503229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e7a48-2843-4cf4-934d-53aabcc33d9e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:37.504013 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e7a48-2843-4cf4-934d-53aabcc33d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:37.677327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b01e7a48-2843-4cf4-934d-53aabcc33d9e None None] MFA Rules not processed for user `1b76876d821e4bbbb8d5cad5bd26f559`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:37.679479 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b01e7a48-2843-4cf4-934d-53aabcc33d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:37.778241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:22:37.807034 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 672/1351] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 06:22:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:22:41.188857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:22:42.854720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952db199-4105-4c5d-86cc-b46eb119e85b None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:42.855630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952db199-4105-4c5d-86cc-b46eb119e85b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:42.855630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952db199-4105-4c5d-86cc-b46eb119e85b None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:42.859247 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 680/1352] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 06:22:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:22:42.865908 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf782eb-8884-473a-9d29-0bebeaa5aa23 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:42.866507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf782eb-8884-473a-9d29-0bebeaa5aa23 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:42.866507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf782eb-8884-473a-9d29-0bebeaa5aa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:42.886678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-2cf782eb-8884-473a-9d29-0bebeaa5aa23 None None] MFA Rules not processed for user `1b76876d821e4bbbb8d5cad5bd26f559`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:42.887896 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2cf782eb-8884-473a-9d29-0bebeaa5aa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:42.898358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 673/1353] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 06:22:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:22:44.684026 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf271b9-90b1-4ab5-9542-44e6e68d14f2 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:44.684784 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf271b9-90b1-4ab5-9542-44e6e68d14f2 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:44.684784 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf271b9-90b1-4ab5-9542-44e6e68d14f2 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:44.685986 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 681/1354] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 06:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:22:44.693078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab93a4-9f48-4196-aca1-0ccdea6fb30d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:44.693623 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab93a4-9f48-4196-aca1-0ccdea6fb30d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:44.694037 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab93a4-9f48-4196-aca1-0ccdea6fb30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:44.723224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9cab93a4-9f48-4196-aca1-0ccdea6fb30d None None] MFA Rules not processed for user `1b76876d821e4bbbb8d5cad5bd26f559`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:44.724474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9cab93a4-9f48-4196-aca1-0ccdea6fb30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:44.736174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 674/1355] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 06:22:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:22:48.715753 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c321ee-5b47-461d-9473-b8a32a46b4e1 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:48.716222 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c321ee-5b47-461d-9473-b8a32a46b4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:48.716330 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c321ee-5b47-461d-9473-b8a32a46b4e1 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:48.717981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 682/1356] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:22:48.725076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7fe7f-0e88-4e98-962b-2ec1b00a64e2 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:48.725670 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7fe7f-0e88-4e98-962b-2ec1b00a64e2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:48.725670 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7fe7f-0e88-4e98-962b-2ec1b00a64e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:48.750294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-25b7fe7f-0e88-4e98-962b-2ec1b00a64e2 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:48.752272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-25b7fe7f-0e88-4e98-962b-2ec1b00a64e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:48.771178 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 675/1357] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:22:48.834133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a6dca-e7ee-4e6f-98db-66e51980823a None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:48.835070 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a6dca-e7ee-4e6f-98db-66e51980823a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:48.835642 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a6dca-e7ee-4e6f-98db-66e51980823a None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:48.837508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 683/1358] 10.222.0.65 () {54 vars in 961 bytes} [Thu May 19 06:22:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:22:48.847147 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e49ab-208f-4396-9001-1b7898f8f404 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:48.848169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e49ab-208f-4396-9001-1b7898f8f404 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:48.848644 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e49ab-208f-4396-9001-1b7898f8f404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:48.930051 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-610e49ab-208f-4396-9001-1b7898f8f404 None None] MFA Rules not processed for user `2a09f4878eac4d60a9a7dcedba15c05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:48.931242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-610e49ab-208f-4396-9001-1b7898f8f404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:49.052641 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 676/1359] 10.222.0.65 () {58 vars in 1058 bytes} [Thu May 19 06:22:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:22:49.067125 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a6dca-e7ee-4e6f-98db-66e51980823a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:22:49.086743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a6dca-e7ee-4e6f-98db-66e51980823a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:22:49.093815 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26186efc-4813-4174-895b-7cfa461ac095 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:22:49.094814 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26186efc-4813-4174-895b-7cfa461ac095 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:49.095485 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26186efc-4813-4174-895b-7cfa461ac095 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:49.096223 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26186efc-4813-4174-895b-7cfa461ac095 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:49.107944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26186efc-4813-4174-895b-7cfa461ac095 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:22:49.110129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26186efc-4813-4174-895b-7cfa461ac095 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:22:49.125689 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 684/1360] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:22:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:22:51.810209 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180a2e9-07a3-4a88-8766-3e0707d1c7b2 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:51.811034 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180a2e9-07a3-4a88-8766-3e0707d1c7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:51.811215 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180a2e9-07a3-4a88-8766-3e0707d1c7b2 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:51.815606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 677/1361] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:22:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:22:51.822637 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4e659-48c3-4a08-95d3-25b2fbbcce33 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:51.823498 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4e659-48c3-4a08-95d3-25b2fbbcce33 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:51.823689 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4e659-48c3-4a08-95d3-25b2fbbcce33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:51.855634 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-77e4e659-48c3-4a08-95d3-25b2fbbcce33 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:51.858006 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-77e4e659-48c3-4a08-95d3-25b2fbbcce33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:51.872952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 685/1362] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:22:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:22:51.939474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180a2e9-07a3-4a88-8766-3e0707d1c7b2 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:22:51.964545 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180a2e9-07a3-4a88-8766-3e0707d1c7b2 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:22:51.974891 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e41090-0a10-4e56-91a4-802712e6f477 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:22:51.976094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e41090-0a10-4e56-91a4-802712e6f477 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:51.976638 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e41090-0a10-4e56-91a4-802712e6f477 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:51.977195 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e41090-0a10-4e56-91a4-802712e6f477 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:51.992556 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e41090-0a10-4e56-91a4-802712e6f477 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:22:51.994732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e41090-0a10-4e56-91a4-802712e6f477 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:22:52.006462 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 678/1363] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:22:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:22:54.838364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fa36f-3cc1-4698-9400-4aab212bfd42 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:54.838840 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fa36f-3cc1-4698-9400-4aab212bfd42 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:54.839089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fa36f-3cc1-4698-9400-4aab212bfd42 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:54.840591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 686/1364] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:22:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:22:54.847557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0151d-c00e-42cc-b5cc-6a3042d4f0e5 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:54.848030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0151d-c00e-42cc-b5cc-6a3042d4f0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:54.848139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0151d-c00e-42cc-b5cc-6a3042d4f0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:54.867161 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f3f0151d-c00e-42cc-b5cc-6a3042d4f0e5 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:54.868361 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f3f0151d-c00e-42cc-b5cc-6a3042d4f0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:54.879636 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 679/1365] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:22:54.932260 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e11cc-842a-4815-ae64-003278c0aca9 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:54.932631 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e11cc-842a-4815-ae64-003278c0aca9 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:54.933079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e11cc-842a-4815-ae64-003278c0aca9 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:54.935272 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 687/1366] 10.222.0.65 () {54 vars in 961 bytes} [Thu May 19 06:22:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:22:54.944034 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c288efa-0503-46eb-b22d-a8402cd6653e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:54.944469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c288efa-0503-46eb-b22d-a8402cd6653e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:54.944879 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c288efa-0503-46eb-b22d-a8402cd6653e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:54.965605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8c288efa-0503-46eb-b22d-a8402cd6653e None None] MFA Rules not processed for user `2a09f4878eac4d60a9a7dcedba15c05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:54.966829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8c288efa-0503-46eb-b22d-a8402cd6653e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:54.977453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 680/1367] 10.222.0.65 () {58 vars in 1058 bytes} [Thu May 19 06:22:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:22:54.988391 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e11cc-842a-4815-ae64-003278c0aca9 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:22:55.003755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e11cc-842a-4815-ae64-003278c0aca9 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:22:55.009243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffe73a-8152-47f1-b46e-3a31851daee0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:22:55.009918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffe73a-8152-47f1-b46e-3a31851daee0 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:55.010211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffe73a-8152-47f1-b46e-3a31851daee0 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:55.010491 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffe73a-8152-47f1-b46e-3a31851daee0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:55.020175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffe73a-8152-47f1-b46e-3a31851daee0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:22:55.021901 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffe73a-8152-47f1-b46e-3a31851daee0 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:22:55.036265 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 688/1368] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:22:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:22:57.529956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79875be7-1b88-49b5-9cb9-8d550df6fc71 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:57.531211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79875be7-1b88-49b5-9cb9-8d550df6fc71 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:57.531211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79875be7-1b88-49b5-9cb9-8d550df6fc71 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:57.532151 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 681/1369] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:22:57.538510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83954d81-21b3-4ca7-9d60-9064934cb791 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:57.539344 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83954d81-21b3-4ca7-9d60-9064934cb791 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:57.539624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83954d81-21b3-4ca7-9d60-9064934cb791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:57.559493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-83954d81-21b3-4ca7-9d60-9064934cb791 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:57.560726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-83954d81-21b3-4ca7-9d60-9064934cb791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:57.570501 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 689/1370] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:22:57.617544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79875be7-1b88-49b5-9cb9-8d550df6fc71 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:22:57.634836 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79875be7-1b88-49b5-9cb9-8d550df6fc71 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:22:57.640750 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d52a3-1be8-48ca-b2ca-81ab5b04634c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:22:57.641432 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d52a3-1be8-48ca-b2ca-81ab5b04634c service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:57.641720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d52a3-1be8-48ca-b2ca-81ab5b04634c service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:57.642004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d52a3-1be8-48ca-b2ca-81ab5b04634c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:57.661896 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d52a3-1be8-48ca-b2ca-81ab5b04634c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:22:57.665827 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d52a3-1be8-48ca-b2ca-81ab5b04634c service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:22:57.687427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 682/1371] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:22:57.896706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53d226-71eb-4c94-9dc5-adb72c4cf31c None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:57.896993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53d226-71eb-4c94-9dc5-adb72c4cf31c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:57.897513 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53d226-71eb-4c94-9dc5-adb72c4cf31c None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:57.899117 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 690/1372] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:22:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:22:57.905141 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0984a-bade-44c0-b97a-1f52f8a71cb9 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:57.906773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0984a-bade-44c0-b97a-1f52f8a71cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:57.906773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0984a-bade-44c0-b97a-1f52f8a71cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:57.925952 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d4a0984a-bade-44c0-b97a-1f52f8a71cb9 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:57.927203 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d4a0984a-bade-44c0-b97a-1f52f8a71cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:57.935721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 683/1373] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:22:58.288252 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81eb30d-fd7c-4400-aa3a-19d484574dbd None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:58.288689 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81eb30d-fd7c-4400-aa3a-19d484574dbd None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:58.288943 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81eb30d-fd7c-4400-aa3a-19d484574dbd None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:58.290340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 691/1374] 10.222.0.65 () {54 vars in 963 bytes} [Thu May 19 06:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:22:58.296484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e98e98-c37d-4a1a-9595-abe7eeb0356a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:58.296782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e98e98-c37d-4a1a-9595-abe7eeb0356a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:58.297081 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e98e98-c37d-4a1a-9595-abe7eeb0356a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:58.365723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a6e98e98-c37d-4a1a-9595-abe7eeb0356a None None] MFA Rules not processed for user `c92d7ec81ddc4da7a33f9dbb756e59b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:22:58.366956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a6e98e98-c37d-4a1a-9595-abe7eeb0356a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:22:58.371540 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:22:58.441426 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 684/1375] 10.222.0.65 () {58 vars in 1060 bytes} [Thu May 19 06:22:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:22:58.453832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81eb30d-fd7c-4400-aa3a-19d484574dbd None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:22:58.472549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81eb30d-fd7c-4400-aa3a-19d484574dbd None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:22:58.480093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa2cf3-2577-4af7-83c0-7da070791d40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:22:58.481266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa2cf3-2577-4af7-83c0-7da070791d40 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:22:58.482019 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa2cf3-2577-4af7-83c0-7da070791d40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:22:58.482563 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa2cf3-2577-4af7-83c0-7da070791d40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:22:58.500327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa2cf3-2577-4af7-83c0-7da070791d40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:22:58.504205 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa2cf3-2577-4af7-83c0-7da070791d40 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:22:58.525352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 692/1376] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:22:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:23:00.286155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62efe3c-a3e7-40f4-a7bc-bcf0fe8184b0 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:00.286857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62efe3c-a3e7-40f4-a7bc-bcf0fe8184b0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:00.286857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62efe3c-a3e7-40f4-a7bc-bcf0fe8184b0 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:00.288337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 685/1377] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:00.294509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda59a8-ec67-40f0-aa47-98621c7cfc00 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:00.295098 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda59a8-ec67-40f0-aa47-98621c7cfc00 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:00.295220 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda59a8-ec67-40f0-aa47-98621c7cfc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:00.316140 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3bda59a8-ec67-40f0-aa47-98621c7cfc00 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:00.317516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3bda59a8-ec67-40f0-aa47-98621c7cfc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:00.331972 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 693/1378] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:23:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:00.380084 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62efe3c-a3e7-40f4-a7bc-bcf0fe8184b0 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:00.400263 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62efe3c-a3e7-40f4-a7bc-bcf0fe8184b0 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:00.406898 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0aa687-6724-47d1-9ef8-7eb18ddea209 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:00.407198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0aa687-6724-47d1-9ef8-7eb18ddea209 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:00.407487 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0aa687-6724-47d1-9ef8-7eb18ddea209 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:00.408435 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0aa687-6724-47d1-9ef8-7eb18ddea209 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:00.418734 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0aa687-6724-47d1-9ef8-7eb18ddea209 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:00.420391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0aa687-6724-47d1-9ef8-7eb18ddea209 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:00.432610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 686/1379] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:02.768906 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ed887-fe48-4569-853f-f03a4c8458b2 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:02.770438 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ed887-fe48-4569-853f-f03a4c8458b2 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:02.771050 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ed887-fe48-4569-853f-f03a4c8458b2 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:02.773418 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 694/1380] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:23:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:02.779628 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80dcfa-3a26-4d61-995c-65a5eca1e8af None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:02.781021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80dcfa-3a26-4d61-995c-65a5eca1e8af None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:02.781655 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80dcfa-3a26-4d61-995c-65a5eca1e8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:02.806913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-eb80dcfa-3a26-4d61-995c-65a5eca1e8af None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:02.808599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-eb80dcfa-3a26-4d61-995c-65a5eca1e8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:02.818839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 687/1381] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:23:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:02.870347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ed887-fe48-4569-853f-f03a4c8458b2 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:02.886574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ed887-fe48-4569-853f-f03a4c8458b2 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:02.893711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b032b2c-8494-4783-ba59-36674e43a5d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:02.894551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b032b2c-8494-4783-ba59-36674e43a5d3 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:02.894936 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b032b2c-8494-4783-ba59-36674e43a5d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:02.895316 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b032b2c-8494-4783-ba59-36674e43a5d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:02.907901 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b032b2c-8494-4783-ba59-36674e43a5d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:02.909793 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b032b2c-8494-4783-ba59-36674e43a5d3 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:02.921830 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 695/1382] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:05.333304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38d85b-73ce-46ad-a35a-218fb396d39f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:05.334204 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38d85b-73ce-46ad-a35a-218fb396d39f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:05.334691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38d85b-73ce-46ad-a35a-218fb396d39f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:05.338232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 688/1383] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:23:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:05.342161 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e29529-5d34-439f-8476-e1451e7f78ac None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:05.342681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e29529-5d34-439f-8476-e1451e7f78ac None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:05.342765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e29529-5d34-439f-8476-e1451e7f78ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:05.364784 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-90e29529-5d34-439f-8476-e1451e7f78ac None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:05.366428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-90e29529-5d34-439f-8476-e1451e7f78ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:05.375133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 696/1384] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:23:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:05.431845 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38d85b-73ce-46ad-a35a-218fb396d39f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:05.448441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38d85b-73ce-46ad-a35a-218fb396d39f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:05.455032 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0de2b-809c-4b7b-9b28-096f3210dd2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:05.455819 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0de2b-809c-4b7b-9b28-096f3210dd2d service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:05.456346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0de2b-809c-4b7b-9b28-096f3210dd2d service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:05.456724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0de2b-809c-4b7b-9b28-096f3210dd2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:05.467085 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca0de2b-809c-4b7b-9b28-096f3210dd2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:05.468944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca0de2b-809c-4b7b-9b28-096f3210dd2d service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:05.479697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 689/1385] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:07.807139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b263e-7361-49eb-a43e-42c7c2608104 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:07.807680 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b263e-7361-49eb-a43e-42c7c2608104 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:07.807680 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b263e-7361-49eb-a43e-42c7c2608104 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:07.809060 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 697/1386] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:07.816389 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb0e38-8e99-494d-b326-55407b25adb5 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:07.816852 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb0e38-8e99-494d-b326-55407b25adb5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:07.817206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb0e38-8e99-494d-b326-55407b25adb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:07.840640 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9cdb0e38-8e99-494d-b326-55407b25adb5 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:07.841928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9cdb0e38-8e99-494d-b326-55407b25adb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:07.852099 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 690/1387] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:07.909112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b263e-7361-49eb-a43e-42c7c2608104 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:07.928480 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b263e-7361-49eb-a43e-42c7c2608104 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:07.936482 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bc4e6-651d-40b9-b534-19b900f7b0d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:07.937425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bc4e6-651d-40b9-b534-19b900f7b0d6 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:07.937955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bc4e6-651d-40b9-b534-19b900f7b0d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:07.938314 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bc4e6-651d-40b9-b534-19b900f7b0d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:07.950405 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bc4e6-651d-40b9-b534-19b900f7b0d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:07.952584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bc4e6-651d-40b9-b534-19b900f7b0d6 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:07.967042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 698/1388] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:10.358650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e07007-2379-4db3-8e89-1fc4765c892d None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:10.359442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e07007-2379-4db3-8e89-1fc4765c892d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:10.360129 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e07007-2379-4db3-8e89-1fc4765c892d None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:10.365031 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 691/1389] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:23:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:10.370592 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5510d5a-3ecf-4172-9d69-d7f438a476a9 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:10.371080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5510d5a-3ecf-4172-9d69-d7f438a476a9 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:10.371181 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5510d5a-3ecf-4172-9d69-d7f438a476a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:10.394185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f5510d5a-3ecf-4172-9d69-d7f438a476a9 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:10.395672 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f5510d5a-3ecf-4172-9d69-d7f438a476a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:10.404093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 699/1390] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:10.450729 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e07007-2379-4db3-8e89-1fc4765c892d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:10.465769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e07007-2379-4db3-8e89-1fc4765c892d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:10.473427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8754dd7-f5c2-4cdb-8329-158e775ee2da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:10.474101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8754dd7-f5c2-4cdb-8329-158e775ee2da service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:10.474379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8754dd7-f5c2-4cdb-8329-158e775ee2da service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:10.474705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8754dd7-f5c2-4cdb-8329-158e775ee2da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:10.484842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8754dd7-f5c2-4cdb-8329-158e775ee2da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:10.487174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8754dd7-f5c2-4cdb-8329-158e775ee2da service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:10.499564 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 692/1391] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:13.297427 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf1231-ca93-4f2b-81c8-93de8fea9815 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:13.298129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf1231-ca93-4f2b-81c8-93de8fea9815 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:13.298129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf1231-ca93-4f2b-81c8-93de8fea9815 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:13.299447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 700/1392] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:23:13.305431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754d24d-d236-4014-92b8-59f71e754524 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:13.306020 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754d24d-d236-4014-92b8-59f71e754524 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:13.306020 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754d24d-d236-4014-92b8-59f71e754524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:13.325389 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f754d24d-d236-4014-92b8-59f71e754524 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:13.326918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f754d24d-d236-4014-92b8-59f71e754524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:13.340369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 693/1393] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:13.386013 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf1231-ca93-4f2b-81c8-93de8fea9815 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:13.394231 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=46 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:23:13.401627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf1231-ca93-4f2b-81c8-93de8fea9815 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:13.408024 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f3190-891f-44b4-b061-03d2fe83ec1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:13.408947 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f3190-891f-44b4-b061-03d2fe83ec1d service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:13.409142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f3190-891f-44b4-b061-03d2fe83ec1d service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:13.409455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f3190-891f-44b4-b061-03d2fe83ec1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:13.418242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f3190-891f-44b4-b061-03d2fe83ec1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:13.419843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f3190-891f-44b4-b061-03d2fe83ec1d service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:13.431065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 701/1394] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:15.381427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9efb4-2964-45a0-affc-b55c9978fc3f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:15.381995 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9efb4-2964-45a0-affc-b55c9978fc3f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:15.381995 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9efb4-2964-45a0-affc-b55c9978fc3f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:15.383382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 694/1395] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:15.389073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f2996-e070-40d9-bdab-bf2b955ef7ba None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:15.389689 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f2996-e070-40d9-bdab-bf2b955ef7ba None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:15.389689 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f2996-e070-40d9-bdab-bf2b955ef7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:15.414373 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f58f2996-e070-40d9-bdab-bf2b955ef7ba None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:15.415592 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f58f2996-e070-40d9-bdab-bf2b955ef7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:15.423572 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 702/1396] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:15.484489 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1472e1-afe8-4d53-a2ad-df5808d7a5a8 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:15.484945 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1472e1-afe8-4d53-a2ad-df5808d7a5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:15.485369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1472e1-afe8-4d53-a2ad-df5808d7a5a8 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:15.487143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 695/1397] 10.222.0.65 () {54 vars in 962 bytes} [Thu May 19 06:23:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:15.492407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68814ace-c501-4f44-a2bb-07298eafc158 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:15.492921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68814ace-c501-4f44-a2bb-07298eafc158 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:15.493275 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68814ace-c501-4f44-a2bb-07298eafc158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:15.544004 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-68814ace-c501-4f44-a2bb-07298eafc158 None None] MFA Rules not processed for user `a812c3b9b20f44f8814051381a8005b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:15.545242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-68814ace-c501-4f44-a2bb-07298eafc158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:15.614049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 703/1398] 10.222.0.65 () {58 vars in 1059 bytes} [Thu May 19 06:23:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:15.621770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1472e1-afe8-4d53-a2ad-df5808d7a5a8 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:15.633629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1472e1-afe8-4d53-a2ad-df5808d7a5a8 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:15.637394 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e3cd8-525c-4222-9d4a-df3471dda2e4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a812c3b9b20f44f8814051381a8005b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:15.638135 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e3cd8-525c-4222-9d4a-df3471dda2e4 service glance] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:15.638508 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e3cd8-525c-4222-9d4a-df3471dda2e4 service glance] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:15.638861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e3cd8-525c-4222-9d4a-df3471dda2e4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:15.649562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e3cd8-525c-4222-9d4a-df3471dda2e4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:15.651116 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e3cd8-525c-4222-9d4a-df3471dda2e4 service glance] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:15.661857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 696/1399] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:15.743630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5397fb-6ce6-4413-bafc-ab810244117f None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:15.743955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5397fb-6ce6-4413-bafc-ab810244117f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:15.744210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5397fb-6ce6-4413-bafc-ab810244117f None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:15.745776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 704/1400] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:15.751025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d991c3-0012-4b32-bb70-3770e70bd5e9 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:15.751942 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d991c3-0012-4b32-bb70-3770e70bd5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:15.752231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d991c3-0012-4b32-bb70-3770e70bd5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:15.770794 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b9d991c3-0012-4b32-bb70-3770e70bd5e9 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:15.771973 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b9d991c3-0012-4b32-bb70-3770e70bd5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:15.779923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 697/1401] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:15.825977 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5397fb-6ce6-4413-bafc-ab810244117f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:15.840473 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5397fb-6ce6-4413-bafc-ab810244117f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:15.846594 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c94461-6a7a-4e97-b79a-32e611be0d7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:15.847253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c94461-6a7a-4e97-b79a-32e611be0d7e service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:15.847539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c94461-6a7a-4e97-b79a-32e611be0d7e service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:15.847820 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c94461-6a7a-4e97-b79a-32e611be0d7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:15.861828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c94461-6a7a-4e97-b79a-32e611be0d7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:15.863492 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c94461-6a7a-4e97-b79a-32e611be0d7e service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:15.874093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 705/1402] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:16.244634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e30cb9-40bf-4315-ab73-b3ee028367a2 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:16.244869 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e30cb9-40bf-4315-ab73-b3ee028367a2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:16.245154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e30cb9-40bf-4315-ab73-b3ee028367a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:16.297553 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-95e30cb9-40bf-4315-ab73-b3ee028367a2 None None] MFA Rules not processed for user `aee77a3ed91445ab8b7f58e7f8bc348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:16.299411 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-95e30cb9-40bf-4315-ab73-b3ee028367a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:16.370942 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 698/1403] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 06:23:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:16.417194 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3522667-4868-437f-8530-7c4a2b3cc8c9 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:16.417717 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3522667-4868-437f-8530-7c4a2b3cc8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:16.417811 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3522667-4868-437f-8530-7c4a2b3cc8c9 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:16.419219 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 706/1404] 10.222.0.65 () {54 vars in 942 bytes} [Thu May 19 06:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:16.426665 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dd99b-70f1-4438-a490-774d9ffd3ccf None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:16.427129 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dd99b-70f1-4438-a490-774d9ffd3ccf None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:16.427315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dd99b-70f1-4438-a490-774d9ffd3ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:16.435366 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:23:16.486473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-da1dd99b-70f1-4438-a490-774d9ffd3ccf None None] MFA Rules not processed for user `5fc205d053d54506bb6c9227e4a1b044`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:16.487800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-da1dd99b-70f1-4438-a490-774d9ffd3ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:16.562470 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 699/1405] 10.222.0.65 () {58 vars in 1039 bytes} [Thu May 19 06:23:16 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:16.572601 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3522667-4868-437f-8530-7c4a2b3cc8c9 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:16.587498 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3522667-4868-437f-8530-7c4a2b3cc8c9 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:16.592702 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95da6d-3d62-4c06-bdfc-f2fb7a4b5da2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc205d053d54506bb6c9227e4a1b044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:16.593644 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95da6d-3d62-4c06-bdfc-f2fb7a4b5da2 service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:16.593818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95da6d-3d62-4c06-bdfc-f2fb7a4b5da2 service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:16.594163 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95da6d-3d62-4c06-bdfc-f2fb7a4b5da2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:16.603210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95da6d-3d62-4c06-bdfc-f2fb7a4b5da2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:16.604896 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95da6d-3d62-4c06-bdfc-f2fb7a4b5da2 service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:16.613769 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 707/1406] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 06:23:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 06:23:18.205818 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aff3c5-3c56-49e1-a60e-76be7c2d0d89 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:18.206673 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aff3c5-3c56-49e1-a60e-76be7c2d0d89 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:18.206673 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aff3c5-3c56-49e1-a60e-76be7c2d0d89 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:18.207959 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 700/1407] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:23:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:23:18.214131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022537a-a45b-4aae-be68-208831f187f1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:18.214674 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022537a-a45b-4aae-be68-208831f187f1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:18.214674 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022537a-a45b-4aae-be68-208831f187f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:18.235690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2022537a-a45b-4aae-be68-208831f187f1 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:18.237035 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2022537a-a45b-4aae-be68-208831f187f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:18.245200 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 708/1408] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:23:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:23:18.290619 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aff3c5-3c56-49e1-a60e-76be7c2d0d89 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:18.305575 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aff3c5-3c56-49e1-a60e-76be7c2d0d89 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:18.311738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a69e6-a888-4729-afc6-010e48611664 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:18.312410 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a69e6-a888-4729-afc6-010e48611664 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:18.312690 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a69e6-a888-4729-afc6-010e48611664 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:18.312971 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a69e6-a888-4729-afc6-010e48611664 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:18.322201 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a69e6-a888-4729-afc6-010e48611664 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:18.323885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a69e6-a888-4729-afc6-010e48611664 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:18.337467 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 701/1409] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:28.231558 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=61 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:23:28.330405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=49 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:23:41.465266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd8056-0011-42f5-9ba4-57066ded649c None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:41.465905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd8056-0011-42f5-9ba4-57066ded649c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:41.465905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd8056-0011-42f5-9ba4-57066ded649c None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:41.467216 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 709/1410] 10.222.0.65 () {54 vars in 918 bytes} [Thu May 19 06:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:41.472869 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77f667-7d66-46cb-a8f3-b01d65e4670b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:41.473415 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77f667-7d66-46cb-a8f3-b01d65e4670b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:41.473520 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77f667-7d66-46cb-a8f3-b01d65e4670b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:41.492275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-2e77f667-7d66-46cb-a8f3-b01d65e4670b None None] MFA Rules not processed for user `1b76876d821e4bbbb8d5cad5bd26f559`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:41.493503 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2e77f667-7d66-46cb-a8f3-b01d65e4670b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:41.502549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 702/1411] 10.222.0.65 () {58 vars in 1015 bytes} [Thu May 19 06:23:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:41.518145 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04711127-77ba-4440-b950-e9c9965bba63 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:41.518328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04711127-77ba-4440-b950-e9c9965bba63 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:41.518606 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04711127-77ba-4440-b950-e9c9965bba63 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:41.520013 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 710/1412] 10.222.0.65 () {54 vars in 918 bytes} [Thu May 19 06:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:41.525274 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9f191-82aa-405d-979f-f131239d7393 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:41.525607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9f191-82aa-405d-979f-f131239d7393 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:41.525934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9f191-82aa-405d-979f-f131239d7393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:41.545489 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-2cc9f191-82aa-405d-979f-f131239d7393 None None] MFA Rules not processed for user `2a09f4878eac4d60a9a7dcedba15c05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:41.546723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2cc9f191-82aa-405d-979f-f131239d7393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:41.557241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 703/1413] 10.222.0.65 () {58 vars in 1015 bytes} [Thu May 19 06:23:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:44.450821 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47304c85-419c-404a-8628-3ce19a4880c6 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:44.451409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47304c85-419c-404a-8628-3ce19a4880c6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:44.451409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47304c85-419c-404a-8628-3ce19a4880c6 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:44.452731 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 06:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:44.458935 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1a58b-1cd5-48c1-804a-a661b335dbf3 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:44.459404 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1a58b-1cd5-48c1-804a-a661b335dbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:44.459507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1a58b-1cd5-48c1-804a-a661b335dbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:44.477682 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-fdf1a58b-1cd5-48c1-804a-a661b335dbf3 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:44.478907 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fdf1a58b-1cd5-48c1-804a-a661b335dbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:44.487396 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 06:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:44.532796 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47304c85-419c-404a-8628-3ce19a4880c6 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:44.548049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47304c85-419c-404a-8628-3ce19a4880c6 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:44.553921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24901bc8-c53f-4368-8ba6-ba9f41c11715 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:44.554817 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24901bc8-c53f-4368-8ba6-ba9f41c11715 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:44.554817 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24901bc8-c53f-4368-8ba6-ba9f41c11715 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:44.555427 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24901bc8-c53f-4368-8ba6-ba9f41c11715 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:44.564652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24901bc8-c53f-4368-8ba6-ba9f41c11715 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:44.566301 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24901bc8-c53f-4368-8ba6-ba9f41c11715 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:44.576207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 712/1416] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:45.689575 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d4bbd-6203-4656-af9e-67f9cb908fea None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:45.690137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d4bbd-6203-4656-af9e-67f9cb908fea None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:45.690137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d4bbd-6203-4656-af9e-67f9cb908fea None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:45.691518 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 06:23:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:45.696683 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4b375-c30f-4983-a0da-12c4710f84ce None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:45.697184 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4b375-c30f-4983-a0da-12c4710f84ce None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:45.697324 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4b375-c30f-4983-a0da-12c4710f84ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:45.716394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a2b4b375-c30f-4983-a0da-12c4710f84ce None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:45.717638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a2b4b375-c30f-4983-a0da-12c4710f84ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:45.726331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 06:23:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:45.888098 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d4bbd-6203-4656-af9e-67f9cb908fea None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:45.902959 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d4bbd-6203-4656-af9e-67f9cb908fea None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:45.910003 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb6a4c-3144-49fb-baee-1423bb00d14f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:45.910639 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb6a4c-3144-49fb-baee-1423bb00d14f service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:45.910926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb6a4c-3144-49fb-baee-1423bb00d14f service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:45.911203 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb6a4c-3144-49fb-baee-1423bb00d14f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:45.919488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb6a4c-3144-49fb-baee-1423bb00d14f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:45.921069 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb6a4c-3144-49fb-baee-1423bb00d14f service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:45.931985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 706/1419] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:46.889493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404d8b-bed6-4327-ac23-46b7d61b4fae None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:46.890071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404d8b-bed6-4327-ac23-46b7d61b4fae None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:46.890071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404d8b-bed6-4327-ac23-46b7d61b4fae None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:46.891524 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 06:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:46.897605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f474f3e-1f23-4062-ada9-dfd917d0dcda None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:46.898081 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f474f3e-1f23-4062-ada9-dfd917d0dcda None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:46.898196 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f474f3e-1f23-4062-ada9-dfd917d0dcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:46.918016 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5f474f3e-1f23-4062-ada9-dfd917d0dcda None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:46.919205 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5f474f3e-1f23-4062-ada9-dfd917d0dcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:46.927684 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 06:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:23:47.076357 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75404d8b-bed6-4327-ac23-46b7d61b4fae None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:47.091628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75404d8b-bed6-4327-ac23-46b7d61b4fae None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:47.097207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c97e4-83f9-41ad-8685-99231f171194 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:47.097887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c97e4-83f9-41ad-8685-99231f171194 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:47.098175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c97e4-83f9-41ad-8685-99231f171194 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:47.098455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c97e4-83f9-41ad-8685-99231f171194 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:47.107128 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c97e4-83f9-41ad-8685-99231f171194 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:47.108822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c97e4-83f9-41ad-8685-99231f171194 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:47.118867 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 715/1422] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:48.067394 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89707a7a-fc33-47ea-820f-27de0e007625 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:48.067963 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89707a7a-fc33-47ea-820f-27de0e007625 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:48.067963 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89707a7a-fc33-47ea-820f-27de0e007625 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:48.069352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 06:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:48.074368 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fffaa2-18c0-4b0a-98ec-35adf4faa093 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:48.074788 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fffaa2-18c0-4b0a-98ec-35adf4faa093 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:48.074930 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fffaa2-18c0-4b0a-98ec-35adf4faa093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:48.093809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-28fffaa2-18c0-4b0a-98ec-35adf4faa093 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:48.095225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-28fffaa2-18c0-4b0a-98ec-35adf4faa093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:48.103064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 06:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:48.260434 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89707a7a-fc33-47ea-820f-27de0e007625 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:48.274777 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89707a7a-fc33-47ea-820f-27de0e007625 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:48.280182 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef03046-009b-4892-b896-11814162f2e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:48.280812 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef03046-009b-4892-b896-11814162f2e1 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:48.281372 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef03046-009b-4892-b896-11814162f2e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:48.281660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef03046-009b-4892-b896-11814162f2e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:48.290654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef03046-009b-4892-b896-11814162f2e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:48.292252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef03046-009b-4892-b896-11814162f2e1 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:48.302380 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 709/1425] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:49.278061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff0061-e711-439b-9cb5-0b791e7c0247 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:49.278593 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff0061-e711-439b-9cb5-0b791e7c0247 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:49.278593 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff0061-e711-439b-9cb5-0b791e7c0247 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:49.279919 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 06:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:49.285883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d14ef7-0937-4301-a663-25a2f53ce89d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:49.286316 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d14ef7-0937-4301-a663-25a2f53ce89d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:49.286460 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d14ef7-0937-4301-a663-25a2f53ce89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:49.305080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-22d14ef7-0937-4301-a663-25a2f53ce89d None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:49.306364 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-22d14ef7-0937-4301-a663-25a2f53ce89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:49.314486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 06:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:49.465862 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff0061-e711-439b-9cb5-0b791e7c0247 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:49.481003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff0061-e711-439b-9cb5-0b791e7c0247 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:49.487098 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7534d5-cf9c-42bb-981e-1a607ec1c1a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:49.487734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7534d5-cf9c-42bb-981e-1a607ec1c1a0 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:49.488020 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7534d5-cf9c-42bb-981e-1a607ec1c1a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:49.488298 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7534d5-cf9c-42bb-981e-1a607ec1c1a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:49.496766 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7534d5-cf9c-42bb-981e-1a607ec1c1a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:49.498412 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7534d5-cf9c-42bb-981e-1a607ec1c1a0 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:49.508273 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 718/1428] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:50.413938 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc7769-141e-4b29-a1df-ef80623e2048 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:50.414474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc7769-141e-4b29-a1df-ef80623e2048 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:50.414599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc7769-141e-4b29-a1df-ef80623e2048 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:50.415889 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 06:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:50.420874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4451e-a256-4468-ab03-5339852f27f5 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:50.421343 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4451e-a256-4468-ab03-5339852f27f5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:50.421460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4451e-a256-4468-ab03-5339852f27f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:50.439885 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-28c4451e-a256-4468-ab03-5339852f27f5 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:50.441092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-28c4451e-a256-4468-ab03-5339852f27f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:50.448982 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 06:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:50.603134 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc7769-141e-4b29-a1df-ef80623e2048 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:50.617681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc7769-141e-4b29-a1df-ef80623e2048 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:50.623137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fd521-7b54-4e42-8ca8-7d41f9d70dce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:50.623785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fd521-7b54-4e42-8ca8-7d41f9d70dce service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:50.624080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fd521-7b54-4e42-8ca8-7d41f9d70dce service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:50.624363 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fd521-7b54-4e42-8ca8-7d41f9d70dce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:50.632902 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9fd521-7b54-4e42-8ca8-7d41f9d70dce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:50.634559 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9fd521-7b54-4e42-8ca8-7d41f9d70dce service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:50.644769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 712/1431] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:51.522466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537179a-1ed9-4dcb-a51c-1c334c14c922 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:51.523040 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537179a-1ed9-4dcb-a51c-1c334c14c922 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:51.523040 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537179a-1ed9-4dcb-a51c-1c334c14c922 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:51.524515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 06:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:51.529609 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14dd05-02d4-4ce3-b99f-9325cb2817ca None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:51.530088 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14dd05-02d4-4ce3-b99f-9325cb2817ca None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:51.530199 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14dd05-02d4-4ce3-b99f-9325cb2817ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:51.550468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5c14dd05-02d4-4ce3-b99f-9325cb2817ca None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:51.551661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5c14dd05-02d4-4ce3-b99f-9325cb2817ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:51.559674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 06:23:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:51.713529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537179a-1ed9-4dcb-a51c-1c334c14c922 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:51.729413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537179a-1ed9-4dcb-a51c-1c334c14c922 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:51.734724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edb396-bf75-4ade-9971-d5d440d638ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:51.735933 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edb396-bf75-4ade-9971-d5d440d638ce service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:51.736243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edb396-bf75-4ade-9971-d5d440d638ce service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:51.736494 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edb396-bf75-4ade-9971-d5d440d638ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:51.745106 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edb396-bf75-4ade-9971-d5d440d638ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:51.746735 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edb396-bf75-4ade-9971-d5d440d638ce service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:51.756980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 721/1434] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:52.650496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f20722-4cda-41b4-af06-d0022817caba None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:52.651094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f20722-4cda-41b4-af06-d0022817caba None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:52.651094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f20722-4cda-41b4-af06-d0022817caba None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:52.652472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 06:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:52.658175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a61cf-325b-4eaa-bfbd-dcf65382356c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:52.658697 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a61cf-325b-4eaa-bfbd-dcf65382356c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:52.658813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a61cf-325b-4eaa-bfbd-dcf65382356c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:52.676916 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b54a61cf-325b-4eaa-bfbd-dcf65382356c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:52.678153 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b54a61cf-325b-4eaa-bfbd-dcf65382356c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:52.686164 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 06:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:52.836915 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f20722-4cda-41b4-af06-d0022817caba None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:52.851212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f20722-4cda-41b4-af06-d0022817caba None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:52.856586 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded6e1d-ba65-4434-b647-43cc4b7e3648 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:52.857232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded6e1d-ba65-4434-b647-43cc4b7e3648 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:52.857556 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded6e1d-ba65-4434-b647-43cc4b7e3648 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:52.857839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded6e1d-ba65-4434-b647-43cc4b7e3648 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:52.867308 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded6e1d-ba65-4434-b647-43cc4b7e3648 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:52.868916 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded6e1d-ba65-4434-b647-43cc4b7e3648 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:52.880114 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 715/1437] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:53.756338 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15539771-d015-4dd0-bc45-669ffc96a5c2 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:53.756926 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15539771-d015-4dd0-bc45-669ffc96a5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:53.756926 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15539771-d015-4dd0-bc45-669ffc96a5c2 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:53.758348 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 06:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:53.763417 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bea9e8-0bd8-4b47-8706-8cbd51f6fd31 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:53.763874 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bea9e8-0bd8-4b47-8706-8cbd51f6fd31 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:53.763986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bea9e8-0bd8-4b47-8706-8cbd51f6fd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:53.782880 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c9bea9e8-0bd8-4b47-8706-8cbd51f6fd31 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:53.784074 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c9bea9e8-0bd8-4b47-8706-8cbd51f6fd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:53.791889 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 06:23:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:53.940566 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15539771-d015-4dd0-bc45-669ffc96a5c2 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:53.954815 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15539771-d015-4dd0-bc45-669ffc96a5c2 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:53.960177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f29d9f-296f-49a1-bbf0-28c957962847 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:53.960813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f29d9f-296f-49a1-bbf0-28c957962847 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:53.961101 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f29d9f-296f-49a1-bbf0-28c957962847 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:53.961413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f29d9f-296f-49a1-bbf0-28c957962847 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:53.970834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f29d9f-296f-49a1-bbf0-28c957962847 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:53.972447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f29d9f-296f-49a1-bbf0-28c957962847 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:53.981911 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 724/1440] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:56.154634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a4fc1-1dd3-46dc-9d0f-15649b5bdf97 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:56.155486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a4fc1-1dd3-46dc-9d0f-15649b5bdf97 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:56.155683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a4fc1-1dd3-46dc-9d0f-15649b5bdf97 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:56.158122 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 06:23:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:23:56.164561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcfbaa-3cd3-48ab-8b1f-fdc7de1dcbca None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:56.165026 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcfbaa-3cd3-48ab-8b1f-fdc7de1dcbca None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:56.165121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcfbaa-3cd3-48ab-8b1f-fdc7de1dcbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:56.184476 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2dfcfbaa-3cd3-48ab-8b1f-fdc7de1dcbca None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:56.185717 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2dfcfbaa-3cd3-48ab-8b1f-fdc7de1dcbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:56.193914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 06:23:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:56.233124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a4fc1-1dd3-46dc-9d0f-15649b5bdf97 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:56.244831 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a4fc1-1dd3-46dc-9d0f-15649b5bdf97 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:56.248383 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7b69d-e16c-4299-931e-dd59bc3456bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:56.249124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7b69d-e16c-4299-931e-dd59bc3456bf service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:56.249346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7b69d-e16c-4299-931e-dd59bc3456bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:56.249640 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7b69d-e16c-4299-931e-dd59bc3456bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:56.258990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7b69d-e16c-4299-931e-dd59bc3456bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:56.260437 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7b69d-e16c-4299-931e-dd59bc3456bf service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:56.270180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 718/1443] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:23:58.680839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978d47c-9e44-45ff-bcc3-63721e4cd34f None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:58.681575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978d47c-9e44-45ff-bcc3-63721e4cd34f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:58.681575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978d47c-9e44-45ff-bcc3-63721e4cd34f None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:58.682831 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 06:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:23:58.688390 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e932b86-0b49-4434-adff-97497bfce96a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:58.688863 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e932b86-0b49-4434-adff-97497bfce96a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:58.688981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e932b86-0b49-4434-adff-97497bfce96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:58.707436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-4e932b86-0b49-4434-adff-97497bfce96a None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:23:58.708648 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4e932b86-0b49-4434-adff-97497bfce96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:23:58.716662 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 06:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:23:58.755770 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978d47c-9e44-45ff-bcc3-63721e4cd34f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:23:58.767382 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978d47c-9e44-45ff-bcc3-63721e4cd34f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:23:58.770990 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006661c-0564-436b-9636-aa33f710a412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:23:58.771628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006661c-0564-436b-9636-aa33f710a412 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:23:58.771916 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006661c-0564-436b-9636-aa33f710a412 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:23:58.772190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006661c-0564-436b-9636-aa33f710a412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:23:58.782397 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0006661c-0564-436b-9636-aa33f710a412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:23:58.784021 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0006661c-0564-436b-9636-aa33f710a412 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:23:58.793827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 727/1446] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:23:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:24:01.350113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63207505-ec5e-4672-bf64-db50429d2e50 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:01.350668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63207505-ec5e-4672-bf64-db50429d2e50 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:01.350668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63207505-ec5e-4672-bf64-db50429d2e50 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:01.352057 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 06:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:24:01.357383 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df42f19-3952-447a-8daf-f56836b4a479 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:01.357803 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df42f19-3952-447a-8daf-f56836b4a479 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:01.358009 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df42f19-3952-447a-8daf-f56836b4a479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:01.376969 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5df42f19-3952-447a-8daf-f56836b4a479 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:24:01.378219 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5df42f19-3952-447a-8daf-f56836b4a479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:24:01.386236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 06:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:24:01.425016 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63207505-ec5e-4672-bf64-db50429d2e50 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:24:01.435633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63207505-ec5e-4672-bf64-db50429d2e50 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:24:01.439898 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c21ba0-6e73-482b-bd52-bb42f5f06d2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:24:01.440537 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c21ba0-6e73-482b-bd52-bb42f5f06d2c service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:01.440828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c21ba0-6e73-482b-bd52-bb42f5f06d2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:01.441109 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c21ba0-6e73-482b-bd52-bb42f5f06d2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:01.450261 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c21ba0-6e73-482b-bd52-bb42f5f06d2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:24:01.451704 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c21ba0-6e73-482b-bd52-bb42f5f06d2c service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:24:01.461804 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 721/1449] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:24:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:24:03.824792 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa7a8b-c9fd-47d5-b9c1-f6292be90ba6 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:03.825387 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa7a8b-c9fd-47d5-b9c1-f6292be90ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:03.825387 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa7a8b-c9fd-47d5-b9c1-f6292be90ba6 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:03.826752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 06:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:24:03.832357 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba793ab-08ac-48ec-97a8-c6b31134941e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:03.832833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba793ab-08ac-48ec-97a8-c6b31134941e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:03.832940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba793ab-08ac-48ec-97a8-c6b31134941e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:03.850931 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3ba793ab-08ac-48ec-97a8-c6b31134941e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:24:03.852123 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3ba793ab-08ac-48ec-97a8-c6b31134941e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:24:03.860454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 06:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:24:03.899073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa7a8b-c9fd-47d5-b9c1-f6292be90ba6 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:24:03.911440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa7a8b-c9fd-47d5-b9c1-f6292be90ba6 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:24:03.914996 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cbabbf-f511-4874-9aff-7f5fe57d8ce8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:24:03.915662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbabbf-f511-4874-9aff-7f5fe57d8ce8 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:03.915981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbabbf-f511-4874-9aff-7f5fe57d8ce8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:03.916264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbabbf-f511-4874-9aff-7f5fe57d8ce8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:03.926116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cbabbf-f511-4874-9aff-7f5fe57d8ce8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:24:03.927568 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cbabbf-f511-4874-9aff-7f5fe57d8ce8 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:24:03.936925 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 730/1452] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:24:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:24:06.227975 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247f2cd-4985-4f3c-ada7-41d91cd0a505 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:06.228589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247f2cd-4985-4f3c-ada7-41d91cd0a505 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:06.228589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247f2cd-4985-4f3c-ada7-41d91cd0a505 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:06.229983 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 06:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:24:06.235688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5486ad-7bb5-40b2-8814-771f78c9fbcf None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:06.236152 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5486ad-7bb5-40b2-8814-771f78c9fbcf None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:06.236258 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5486ad-7bb5-40b2-8814-771f78c9fbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:06.253648 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5c5486ad-7bb5-40b2-8814-771f78c9fbcf None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:24:06.254826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5c5486ad-7bb5-40b2-8814-771f78c9fbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:24:06.262558 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 06:24:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:24:06.300788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247f2cd-4985-4f3c-ada7-41d91cd0a505 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:24:06.311493 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247f2cd-4985-4f3c-ada7-41d91cd0a505 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:24:06.315780 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7576f8-9974-45ca-8e35-6871c566c936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:24:06.316411 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7576f8-9974-45ca-8e35-6871c566c936 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:06.316698 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7576f8-9974-45ca-8e35-6871c566c936 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:06.316978 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7576f8-9974-45ca-8e35-6871c566c936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:06.326273 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7576f8-9974-45ca-8e35-6871c566c936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:24:06.327709 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7576f8-9974-45ca-8e35-6871c566c936 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:24:06.337953 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 724/1455] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:24:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:24:08.635973 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcded9f9-b22e-4268-8b49-c66f977c95d5 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:08.636533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcded9f9-b22e-4268-8b49-c66f977c95d5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:08.636533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcded9f9-b22e-4268-8b49-c66f977c95d5 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:08.637941 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 06:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:24:08.643391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae31e1-facb-470b-8b81-58d741c4d20c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:08.644006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae31e1-facb-470b-8b81-58d741c4d20c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:08.644006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae31e1-facb-470b-8b81-58d741c4d20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:08.650036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=70 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:24:08.663030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-67ae31e1-facb-470b-8b81-58d741c4d20c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:24:08.664231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-67ae31e1-facb-470b-8b81-58d741c4d20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:24:08.671957 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 06:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:24:08.710946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcded9f9-b22e-4268-8b49-c66f977c95d5 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:24:08.716402 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=64 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:24:08.723576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcded9f9-b22e-4268-8b49-c66f977c95d5 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:24:08.727077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c164c-a964-4c5b-a06a-a4ff9aa108e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:24:08.727720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c164c-a964-4c5b-a06a-a4ff9aa108e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:08.728006 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c164c-a964-4c5b-a06a-a4ff9aa108e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:08.728283 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c164c-a964-4c5b-a06a-a4ff9aa108e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:08.738396 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c164c-a964-4c5b-a06a-a4ff9aa108e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:24:08.739841 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c164c-a964-4c5b-a06a-a4ff9aa108e6 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:24:08.749749 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 733/1458] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:24:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:24:11.056041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b747b28-30c9-4873-a87a-93d21de944d3 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:11.056590 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b747b28-30c9-4873-a87a-93d21de944d3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:11.056716 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b747b28-30c9-4873-a87a-93d21de944d3 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:11.058130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 06:24:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:24:11.064951 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071df49b-9410-4749-94fe-6765872c060c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:11.065457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071df49b-9410-4749-94fe-6765872c060c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:11.065564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071df49b-9410-4749-94fe-6765872c060c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:11.083663 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-071df49b-9410-4749-94fe-6765872c060c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:24:11.084867 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-071df49b-9410-4749-94fe-6765872c060c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:24:11.093131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 06:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:24:11.129542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b747b28-30c9-4873-a87a-93d21de944d3 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:24:11.140363 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b747b28-30c9-4873-a87a-93d21de944d3 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:24:11.144649 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ac665-6cb5-4f24-911f-8c01863164a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:24:11.145337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ac665-6cb5-4f24-911f-8c01863164a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:11.145624 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ac665-6cb5-4f24-911f-8c01863164a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:11.145901 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ac665-6cb5-4f24-911f-8c01863164a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:11.154848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5ac665-6cb5-4f24-911f-8c01863164a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:24:11.156536 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5ac665-6cb5-4f24-911f-8c01863164a1 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:24:11.165964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 727/1461] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:24:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:24:13.438517 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e85c5-5830-4abb-87e3-c8f972323d7a None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:13.439162 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e85c5-5830-4abb-87e3-c8f972323d7a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:13.439162 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e85c5-5830-4abb-87e3-c8f972323d7a None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:13.440556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 06:24:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:24:13.446382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca4827-fc5b-439a-8ddb-14bd500f33b7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:13.446853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca4827-fc5b-439a-8ddb-14bd500f33b7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:13.447012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca4827-fc5b-439a-8ddb-14bd500f33b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:13.465149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d0ca4827-fc5b-439a-8ddb-14bd500f33b7 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:24:13.466365 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d0ca4827-fc5b-439a-8ddb-14bd500f33b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:24:13.474107 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 06:24:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:24:13.511206 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e85c5-5830-4abb-87e3-c8f972323d7a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:24:13.522587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e85c5-5830-4abb-87e3-c8f972323d7a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:24:13.526104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce15f52-0dc4-4a08-b626-f0378c5eddca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:24:13.526748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce15f52-0dc4-4a08-b626-f0378c5eddca service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:13.527032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce15f52-0dc4-4a08-b626-f0378c5eddca service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:13.527310 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce15f52-0dc4-4a08-b626-f0378c5eddca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:13.538521 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce15f52-0dc4-4a08-b626-f0378c5eddca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:24:13.539971 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce15f52-0dc4-4a08-b626-f0378c5eddca service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:24:13.550738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 736/1464] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:24:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:24:15.889546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6816e-659b-4d55-a2a4-19d2d02b8311 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:15.890164 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6816e-659b-4d55-a2a4-19d2d02b8311 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:15.890164 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6816e-659b-4d55-a2a4-19d2d02b8311 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:15.892121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 06:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:24:15.897852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f4ecb-5c1a-4770-9987-b8a9fb0a1958 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:15.898324 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f4ecb-5c1a-4770-9987-b8a9fb0a1958 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:15.898443 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f4ecb-5c1a-4770-9987-b8a9fb0a1958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:15.916256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-914f4ecb-5c1a-4770-9987-b8a9fb0a1958 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:24:15.917480 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-914f4ecb-5c1a-4770-9987-b8a9fb0a1958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:24:15.925881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 06:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:24:15.962365 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6816e-659b-4d55-a2a4-19d2d02b8311 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:24:15.974580 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6816e-659b-4d55-a2a4-19d2d02b8311 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:24:15.978234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f92032-de0c-4641-a687-99e581ed8842 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:24:15.978888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f92032-de0c-4641-a687-99e581ed8842 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:15.979172 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f92032-de0c-4641-a687-99e581ed8842 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:15.979480 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f92032-de0c-4641-a687-99e581ed8842 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:15.990279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f92032-de0c-4641-a687-99e581ed8842 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:24:15.991732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f92032-de0c-4641-a687-99e581ed8842 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:24:16.002114 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 730/1467] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:24:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:24:18.251211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbcc07-7146-4f7e-a1af-b05bcd5eebbd None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:18.251781 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbcc07-7146-4f7e-a1af-b05bcd5eebbd None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:18.251781 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbcc07-7146-4f7e-a1af-b05bcd5eebbd None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:18.253156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 06:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:24:18.258492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ee3eb-ad08-49fa-ae34-30a3ef9be50c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:18.258912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ee3eb-ad08-49fa-ae34-30a3ef9be50c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:18.259066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ee3eb-ad08-49fa-ae34-30a3ef9be50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:18.277571 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e97ee3eb-ad08-49fa-ae34-30a3ef9be50c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:24:18.278565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e97ee3eb-ad08-49fa-ae34-30a3ef9be50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:24:18.288293 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 06:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:24:18.323978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbcc07-7146-4f7e-a1af-b05bcd5eebbd None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:24:18.343265 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbcc07-7146-4f7e-a1af-b05bcd5eebbd None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:24:18.347765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a001eb-7bac-4e3a-84b5-c7f6b26b7ccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:24:18.348415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a001eb-7bac-4e3a-84b5-c7f6b26b7ccf service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:18.348704 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a001eb-7bac-4e3a-84b5-c7f6b26b7ccf service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:18.348986 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a001eb-7bac-4e3a-84b5-c7f6b26b7ccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:18.358388 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a001eb-7bac-4e3a-84b5-c7f6b26b7ccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:24:18.359902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a001eb-7bac-4e3a-84b5-c7f6b26b7ccf service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:24:18.369575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 739/1470] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:24:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:24:28.272011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=20 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:24:28.365115 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=23 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:24:51.880736 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d898e3e-0ce2-4740-9b0b-0a843c2795ad None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:51.881386 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d898e3e-0ce2-4740-9b0b-0a843c2795ad None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:51.881386 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d898e3e-0ce2-4740-9b0b-0a843c2795ad None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:51.882782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 732/1471] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:24:51.888886 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96a824-f30a-459f-ad53-a3a4a7f56587 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:51.889477 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96a824-f30a-459f-ad53-a3a4a7f56587 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:51.889477 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96a824-f30a-459f-ad53-a3a4a7f56587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:51.909729 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-bb96a824-f30a-459f-ad53-a3a4a7f56587 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:24:51.910953 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bb96a824-f30a-459f-ad53-a3a4a7f56587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:24:51.918945 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 740/1472] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:24:51.966278 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be19b7f-314e-4bc3-90e6-6d157ee742a0 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:51.966575 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be19b7f-314e-4bc3-90e6-6d157ee742a0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:51.966862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be19b7f-314e-4bc3-90e6-6d157ee742a0 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:51.968841 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 733/1473] 10.222.0.65 () {54 vars in 962 bytes} [Thu May 19 06:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:24:51.973967 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdee9f6-8bc8-4a8d-98fa-cc734eaf325c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:51.974262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdee9f6-8bc8-4a8d-98fa-cc734eaf325c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:51.974543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdee9f6-8bc8-4a8d-98fa-cc734eaf325c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:51.993165 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2cdee9f6-8bc8-4a8d-98fa-cc734eaf325c None None] MFA Rules not processed for user `a812c3b9b20f44f8814051381a8005b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:24:51.994409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2cdee9f6-8bc8-4a8d-98fa-cc734eaf325c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:24:52.000988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 741/1474] 10.222.0.65 () {58 vars in 1059 bytes} [Thu May 19 06:24:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:24:52.008210 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be19b7f-314e-4bc3-90e6-6d157ee742a0 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:24:52.019545 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be19b7f-314e-4bc3-90e6-6d157ee742a0 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:24:52.023123 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790833e-855e-452b-b29a-93d7df4edd3a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a812c3b9b20f44f8814051381a8005b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:24:52.023758 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790833e-855e-452b-b29a-93d7df4edd3a service glance] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:52.024043 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790833e-855e-452b-b29a-93d7df4edd3a service glance] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:52.024324 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790833e-855e-452b-b29a-93d7df4edd3a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:52.034171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790833e-855e-452b-b29a-93d7df4edd3a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:24:52.035612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790833e-855e-452b-b29a-93d7df4edd3a service glance] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:24:52.045090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 734/1475] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:24:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:24:54.364531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b2efd-4ff8-4da7-9595-7bc0b5d08125 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:54.365073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b2efd-4ff8-4da7-9595-7bc0b5d08125 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:54.365073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b2efd-4ff8-4da7-9595-7bc0b5d08125 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:54.366490 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 742/1476] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:24:54.371782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab34c0-5ca2-44f1-834b-df5a1e6aed39 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:54.372218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab34c0-5ca2-44f1-834b-df5a1e6aed39 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:54.372353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab34c0-5ca2-44f1-834b-df5a1e6aed39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:54.390760 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e0ab34c0-5ca2-44f1-834b-df5a1e6aed39 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:24:54.391934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e0ab34c0-5ca2-44f1-834b-df5a1e6aed39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:24:54.399680 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 735/1477] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:24:54.422888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b2efd-4ff8-4da7-9595-7bc0b5d08125 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:24:54.435329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b2efd-4ff8-4da7-9595-7bc0b5d08125 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:24:54.440789 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a83208-6599-4e2b-b192-9b508d00ae42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:24:54.441481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a83208-6599-4e2b-b192-9b508d00ae42 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:54.441766 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a83208-6599-4e2b-b192-9b508d00ae42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:54.442037 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a83208-6599-4e2b-b192-9b508d00ae42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:54.442765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a83208-6599-4e2b-b192-9b508d00ae42 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:24:54.444539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: WARNING py.warnings [None req-f7a83208-6599-4e2b-b192-9b508d00ae42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 06:24:54.444539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: warnings.warn(msg) May 19 06:24:54.444539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: May 19 06:24:54.445713 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a83208-6599-4e2b-b192-9b508d00ae42 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:24:54.465151 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 743/1478] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 06:24:54 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:24:56.718811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b08fa-70eb-4753-aa69-650a2095966f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:56.719390 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b08fa-70eb-4753-aa69-650a2095966f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:56.719390 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b08fa-70eb-4753-aa69-650a2095966f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:56.720743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 736/1479] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:24:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:24:56.726623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae58fc-cec3-4095-9bcc-12c01bc694f0 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:56.727107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae58fc-cec3-4095-9bcc-12c01bc694f0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:56.727204 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae58fc-cec3-4095-9bcc-12c01bc694f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:56.745819 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1bae58fc-cec3-4095-9bcc-12c01bc694f0 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:24:56.747059 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1bae58fc-cec3-4095-9bcc-12c01bc694f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:24:56.755878 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 744/1480] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:24:56.800335 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b08fa-70eb-4753-aa69-650a2095966f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:24:56.814808 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b08fa-70eb-4753-aa69-650a2095966f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:24:56.820930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae94b08-8adb-4fca-8e0a-6283de4a9602 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:24:56.821724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae94b08-8adb-4fca-8e0a-6283de4a9602 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:56.822106 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae94b08-8adb-4fca-8e0a-6283de4a9602 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:56.822482 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae94b08-8adb-4fca-8e0a-6283de4a9602 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:56.831300 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae94b08-8adb-4fca-8e0a-6283de4a9602 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:24:56.832886 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae94b08-8adb-4fca-8e0a-6283de4a9602 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:24:56.842407 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 737/1481] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:24:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:24:59.267441 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b9db0-9682-4960-a442-6f8deb2bbceb None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:59.268246 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b9db0-9682-4960-a442-6f8deb2bbceb None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:59.268427 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b9db0-9682-4960-a442-6f8deb2bbceb None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:59.269909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 745/1482] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:24:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:24:59.275641 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e0ef6-59be-4a88-aba9-d9ba1972083e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:59.276201 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e0ef6-59be-4a88-aba9-d9ba1972083e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:59.276201 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e0ef6-59be-4a88-aba9-d9ba1972083e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:59.294677 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-657e0ef6-59be-4a88-aba9-d9ba1972083e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:24:59.295874 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-657e0ef6-59be-4a88-aba9-d9ba1972083e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:24:59.303980 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 738/1483] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:24:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:24:59.340666 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b9db0-9682-4960-a442-6f8deb2bbceb None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:24:59.351807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b9db0-9682-4960-a442-6f8deb2bbceb None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:24:59.355299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f32fea-14ff-41b4-9956-2aca1383f767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:24:59.355970 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f32fea-14ff-41b4-9956-2aca1383f767 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:24:59.356258 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f32fea-14ff-41b4-9956-2aca1383f767 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:24:59.356539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f32fea-14ff-41b4-9956-2aca1383f767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:24:59.365644 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f32fea-14ff-41b4-9956-2aca1383f767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:24:59.367092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f32fea-14ff-41b4-9956-2aca1383f767 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:24:59.377216 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 746/1484] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:24:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:25:01.658576 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b110cc0-9a4c-42cb-84fc-e78f1fa8d142 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:25:01.659143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b110cc0-9a4c-42cb-84fc-e78f1fa8d142 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:25:01.659252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b110cc0-9a4c-42cb-84fc-e78f1fa8d142 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:25:01.660644 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 739/1485] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:25:01.666405 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d10d2e-3fc4-48b1-93f2-cf3d2dfcc696 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:25:01.666912 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d10d2e-3fc4-48b1-93f2-cf3d2dfcc696 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:25:01.667027 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d10d2e-3fc4-48b1-93f2-cf3d2dfcc696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:25:01.685953 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c9d10d2e-3fc4-48b1-93f2-cf3d2dfcc696 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:25:01.687202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c9d10d2e-3fc4-48b1-93f2-cf3d2dfcc696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:25:01.695349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 747/1486] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:25:01.734206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b110cc0-9a4c-42cb-84fc-e78f1fa8d142 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:25:01.745478 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b110cc0-9a4c-42cb-84fc-e78f1fa8d142 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:25:01.749204 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5736d-f3f7-4e49-9fc3-b630e811d167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:25:01.749920 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5736d-f3f7-4e49-9fc3-b630e811d167 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:25:01.750206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5736d-f3f7-4e49-9fc3-b630e811d167 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:25:01.750486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5736d-f3f7-4e49-9fc3-b630e811d167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:25:01.760310 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5736d-f3f7-4e49-9fc3-b630e811d167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:25:01.761842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5736d-f3f7-4e49-9fc3-b630e811d167 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:25:01.773016 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 740/1487] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:25:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:25:04.211452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe71b52-2286-43f7-8db9-7993f65e7076 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:25:04.212071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe71b52-2286-43f7-8db9-7993f65e7076 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:25:04.212071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe71b52-2286-43f7-8db9-7993f65e7076 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:25:04.213478 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 748/1488] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:25:04.219624 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdbe5e-17c1-4c4d-8491-0cc2ae745907 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:25:04.220416 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdbe5e-17c1-4c4d-8491-0cc2ae745907 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:25:04.220416 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdbe5e-17c1-4c4d-8491-0cc2ae745907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:25:04.238696 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-45fdbe5e-17c1-4c4d-8491-0cc2ae745907 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:25:04.239918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-45fdbe5e-17c1-4c4d-8491-0cc2ae745907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:25:04.248082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 741/1489] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:25:04.288735 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe71b52-2286-43f7-8db9-7993f65e7076 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:25:04.299843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe71b52-2286-43f7-8db9-7993f65e7076 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:25:04.303358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b192e-3cbc-4a7f-8be8-2e96f9e5fc93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:25:04.303997 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b192e-3cbc-4a7f-8be8-2e96f9e5fc93 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:25:04.304284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b192e-3cbc-4a7f-8be8-2e96f9e5fc93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:25:04.304566 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b192e-3cbc-4a7f-8be8-2e96f9e5fc93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:25:04.314472 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b192e-3cbc-4a7f-8be8-2e96f9e5fc93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:25:04.315892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b192e-3cbc-4a7f-8be8-2e96f9e5fc93 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:25:04.326953 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 749/1490] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:25:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:25:06.843674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8209ae-c4b0-4fcd-9ae2-df744c20c68f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:25:06.844260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8209ae-c4b0-4fcd-9ae2-df744c20c68f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:25:06.844260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8209ae-c4b0-4fcd-9ae2-df744c20c68f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:25:06.845646 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 742/1491] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 06:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:25:06.850959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d39455-0354-46c5-aaaa-33f4cde232bf None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:25:06.851384 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d39455-0354-46c5-aaaa-33f4cde232bf None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:25:06.851527 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d39455-0354-46c5-aaaa-33f4cde232bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:25:06.869188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-80d39455-0354-46c5-aaaa-33f4cde232bf None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:25:06.870431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-80d39455-0354-46c5-aaaa-33f4cde232bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:25:06.879128 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 750/1492] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 06:25:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:25:06.918286 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8209ae-c4b0-4fcd-9ae2-df744c20c68f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:25:06.929774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8209ae-c4b0-4fcd-9ae2-df744c20c68f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:25:06.933565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c5d3f-3bb7-42e5-a196-e91f6f4d2e46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:25:06.934209 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c5d3f-3bb7-42e5-a196-e91f6f4d2e46 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:25:06.934497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c5d3f-3bb7-42e5-a196-e91f6f4d2e46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:25:06.934779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c5d3f-3bb7-42e5-a196-e91f6f4d2e46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:25:06.944847 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c5d3f-3bb7-42e5-a196-e91f6f4d2e46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:25:06.946334 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c5d3f-3bb7-42e5-a196-e91f6f4d2e46 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:25:06.956940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 743/1493] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:25:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:25:16.864061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=33 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:25:16.952581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=33 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:26:48.497815 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaac5bf-47ec-4200-b52b-213674253e01 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:26:48.498385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaac5bf-47ec-4200-b52b-213674253e01 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:26:48.498385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaac5bf-47ec-4200-b52b-213674253e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:26:48.517373 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4aaac5bf-47ec-4200-b52b-213674253e01 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:26:48.518620 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4aaac5bf-47ec-4200-b52b-213674253e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:26:48.526727 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 751/1494] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:26:48.536974 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1fc4b-0ef8-4ac7-9b2b-3d35a0985a23 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:26:48.537545 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1fc4b-0ef8-4ac7-9b2b-3d35a0985a23 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:26:48.537545 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1fc4b-0ef8-4ac7-9b2b-3d35a0985a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:26:48.556732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-27f1fc4b-0ef8-4ac7-9b2b-3d35a0985a23 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:26:48.557960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-27f1fc4b-0ef8-4ac7-9b2b-3d35a0985a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:26:48.565860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 744/1495] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:26:48.572016 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaac5bf-47ec-4200-b52b-213674253e01 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:26:48.582489 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaac5bf-47ec-4200-b52b-213674253e01 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:26:48.587173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c3f82-2896-4d29-8545-907d314781d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:26:48.587842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c3f82-2896-4d29-8545-907d314781d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:26:48.588129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c3f82-2896-4d29-8545-907d314781d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:26:48.588410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c3f82-2896-4d29-8545-907d314781d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:26:48.589168 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c3f82-2896-4d29-8545-907d314781d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:26:48.589507 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c3f82-2896-4d29-8545-907d314781d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:26:48.591266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: WARNING py.warnings [None req-b50c3f82-2896-4d29-8545-907d314781d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 06:26:48.591266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: warnings.warn(msg) May 19 06:26:48.591266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: May 19 06:26:48.592162 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c3f82-2896-4d29-8545-907d314781d5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:26:48.604946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 752/1496] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:26:48.613775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1fc4b-0ef8-4ac7-9b2b-3d35a0985a23 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:26:48.623654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1fc4b-0ef8-4ac7-9b2b-3d35a0985a23 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:26:48.628269 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f11fc7-7510-4c29-9706-0eba15c8e792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:26:48.628922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f11fc7-7510-4c29-9706-0eba15c8e792 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:26:48.629206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f11fc7-7510-4c29-9706-0eba15c8e792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:26:48.629541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f11fc7-7510-4c29-9706-0eba15c8e792 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:26:48.630247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f11fc7-7510-4c29-9706-0eba15c8e792 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:26:48.632059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: WARNING py.warnings [None req-f6f11fc7-7510-4c29-9706-0eba15c8e792 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 06:26:48.632059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: warnings.warn(msg) May 19 06:26:48.632059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: May 19 06:26:48.633242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f11fc7-7510-4c29-9706-0eba15c8e792 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:26:48.673056 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 745/1497] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:26:48 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:26:48.677659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c3f82-2896-4d29-8545-907d314781d5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:26:48.688629 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c3f82-2896-4d29-8545-907d314781d5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:26:48.758829 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf402f00-2bbd-4a3c-9312-983c1290afb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:26:48.759437 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf402f00-2bbd-4a3c-9312-983c1290afb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:26:48.759888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf402f00-2bbd-4a3c-9312-983c1290afb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:26:48.760016 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf402f00-2bbd-4a3c-9312-983c1290afb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:26:48.760700 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf402f00-2bbd-4a3c-9312-983c1290afb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:26:48.762116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: WARNING py.warnings [None req-bf402f00-2bbd-4a3c-9312-983c1290afb8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 06:26:48.762116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: warnings.warn(msg) May 19 06:26:48.762116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: May 19 06:26:48.763245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf402f00-2bbd-4a3c-9312-983c1290afb8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:26:48.786461 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 753/1498] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:26:48 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:26:48.790847 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f11fc7-7510-4c29-9706-0eba15c8e792 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:26:48.802376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f11fc7-7510-4c29-9706-0eba15c8e792 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:26:48.807145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd72c06-df4c-4a32-a0d0-2167c5a5aa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:26:48.807820 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd72c06-df4c-4a32-a0d0-2167c5a5aa2a admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:26:48.808101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd72c06-df4c-4a32-a0d0-2167c5a5aa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:26:48.808382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd72c06-df4c-4a32-a0d0-2167c5a5aa2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:26:48.810712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd72c06-df4c-4a32-a0d0-2167c5a5aa2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:26:48.810712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: WARNING py.warnings [None req-3fd72c06-df4c-4a32-a0d0-2167c5a5aa2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 06:26:48.810712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: warnings.warn(msg) May 19 06:26:48.810712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: May 19 06:26:48.811166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd72c06-df4c-4a32-a0d0-2167c5a5aa2a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:26:48.824448 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 746/1499] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:26:48.829175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf402f00-2bbd-4a3c-9312-983c1290afb8 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:26:48.839476 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf402f00-2bbd-4a3c-9312-983c1290afb8 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:26:48.845308 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a59ef8-c3d9-447c-906f-d3dac3e5d77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:26:48.846009 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a59ef8-c3d9-447c-906f-d3dac3e5d77f admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:26:48.846305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a59ef8-c3d9-447c-906f-d3dac3e5d77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:26:48.846587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a59ef8-c3d9-447c-906f-d3dac3e5d77f admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:26:48.847333 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a59ef8-c3d9-447c-906f-d3dac3e5d77f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:26:48.848366 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: WARNING py.warnings [None req-b0a59ef8-c3d9-447c-906f-d3dac3e5d77f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 06:26:48.848366 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: warnings.warn(msg) May 19 06:26:48.848366 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: May 19 06:26:48.849255 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a59ef8-c3d9-447c-906f-d3dac3e5d77f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:26:48.864447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 754/1500] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:26:48.868685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd72c06-df4c-4a32-a0d0-2167c5a5aa2a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:26:48.878876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd72c06-df4c-4a32-a0d0-2167c5a5aa2a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:26:48.883439 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a630f36-9b8a-4351-80a5-b7d0face4b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:26:48.884327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a630f36-9b8a-4351-80a5-b7d0face4b98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:26:48.884615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a630f36-9b8a-4351-80a5-b7d0face4b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:26:48.884894 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a630f36-9b8a-4351-80a5-b7d0face4b98 admin admin] PATH_INFO: `/v3/projects/d290af33209043d2a319c472d8e3ed4a/users/384d714dbdea4df183b2b98e1c56ee95/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:26:48.900372 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a630f36-9b8a-4351-80a5-b7d0face4b98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d290af33209043d2a319c472d8e3ed4a, user_id=384d714dbdea4df183b2b98e1c56ee95, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:26:48.901456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: WARNING py.warnings [None req-6a630f36-9b8a-4351-80a5-b7d0face4b98 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 06:26:48.901456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: warnings.warn(msg) May 19 06:26:48.901456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: May 19 06:26:48.903948 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a630f36-9b8a-4351-80a5-b7d0face4b98 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:26:48.917382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 747/1501] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:26:48 2022] PUT /identity/v3/projects/d290af33209043d2a319c472d8e3ed4a/users/384d714dbdea4df183b2b98e1c56ee95/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:26:48.928699 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3696b92-5be8-47bd-8dad-c9dce1da8eca None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:26:48.928993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3696b92-5be8-47bd-8dad-c9dce1da8eca None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:26:48.929274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3696b92-5be8-47bd-8dad-c9dce1da8eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:26:48.957124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b3696b92-5be8-47bd-8dad-c9dce1da8eca None None] MFA Rules not processed for user `384d714dbdea4df183b2b98e1c56ee95`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:26:48.958425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b3696b92-5be8-47bd-8dad-c9dce1da8eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:26:49.024275 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 755/1502] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:26:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:26:49.038498 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52901cd7-1896-42dc-beb5-d132c93871e1 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:26:49.038799 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52901cd7-1896-42dc-beb5-d132c93871e1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:26:49.039082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52901cd7-1896-42dc-beb5-d132c93871e1 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:26:49.040491 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 748/1503] 10.222.0.65 () {54 vars in 963 bytes} [Thu May 19 06:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:26:49.045490 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72346d-9dfa-4879-ad0a-e95dacd5ea97 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:26:49.045787 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72346d-9dfa-4879-ad0a-e95dacd5ea97 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:26:49.046064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72346d-9dfa-4879-ad0a-e95dacd5ea97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:26:49.064119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-6a72346d-9dfa-4879-ad0a-e95dacd5ea97 None None] MFA Rules not processed for user `c92d7ec81ddc4da7a33f9dbb756e59b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:26:49.065460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6a72346d-9dfa-4879-ad0a-e95dacd5ea97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:26:49.110253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:26:49.122723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 756/1504] 10.222.0.65 () {58 vars in 1060 bytes} [Thu May 19 06:26:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:26:49.130542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52901cd7-1896-42dc-beb5-d132c93871e1 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:26:49.142381 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52901cd7-1896-42dc-beb5-d132c93871e1 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:26:49.146137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53690d-306f-4ade-a096-abd0a82462e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:26:49.146777 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53690d-306f-4ade-a096-abd0a82462e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:26:49.147061 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53690d-306f-4ade-a096-abd0a82462e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:26:49.147339 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53690d-306f-4ade-a096-abd0a82462e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:26:49.156994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53690d-306f-4ade-a096-abd0a82462e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:26:49.158472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53690d-306f-4ade-a096-abd0a82462e6 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:26:49.167429 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 749/1505] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:26:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:26:49.210034 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d68b8d-420c-488c-9a39-0f51915c296c None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:26:49.210326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d68b8d-420c-488c-9a39-0f51915c296c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:26:49.210606 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d68b8d-420c-488c-9a39-0f51915c296c None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:26:49.211900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 757/1506] 10.222.0.65 () {52 vars in 829 bytes} [Thu May 19 06:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:26:49.216148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53690d-306f-4ade-a096-abd0a82462e6 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:26:49.285934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53690d-306f-4ade-a096-abd0a82462e6 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:26:49.290353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb33de4-c4ab-4431-bb32-d17453abc4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:26:49.291218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb33de4-c4ab-4431-bb32-d17453abc4a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:26:49.291497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb33de4-c4ab-4431-bb32-d17453abc4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:26:49.291772 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb33de4-c4ab-4431-bb32-d17453abc4a6 admin admin] PATH_INFO: `/v3/users/384d714dbdea4df183b2b98e1c56ee95` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:26:49.292983 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb33de4-c4ab-4431-bb32-d17453abc4a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=384d714dbdea4df183b2b98e1c56ee95)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:26:49.294110 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: WARNING py.warnings [None req-4eb33de4-c4ab-4431-bb32-d17453abc4a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 06:26:49.294110 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: warnings.warn(msg) May 19 06:26:49.294110 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: May 19 06:26:49.295299 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb33de4-c4ab-4431-bb32-d17453abc4a6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:26:49.366933 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4eb33de4-c4ab-4431-bb32-d17453abc4a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '384d714dbdea4df183b2b98e1c56ee95'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:26:49.367667 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4eb33de4-c4ab-4431-bb32-d17453abc4a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '384d714dbdea4df183b2b98e1c56ee95'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:26:49.388744 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4eb33de4-c4ab-4431-bb32-d17453abc4a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '384d714dbdea4df183b2b98e1c56ee95'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:26:49.389897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4eb33de4-c4ab-4431-bb32-d17453abc4a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '384d714dbdea4df183b2b98e1c56ee95'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:26:49.402756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4eb33de4-c4ab-4431-bb32-d17453abc4a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '384d714dbdea4df183b2b98e1c56ee95'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:26:49.412852 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:26:49.425063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 750/1507] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:26:49 2022] DELETE /identity/v3/users/384d714dbdea4df183b2b98e1c56ee95 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:26:49.429637 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8ccf5-546c-4d27-80b4-17e65cc499b0 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:26:49.429933 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8ccf5-546c-4d27-80b4-17e65cc499b0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:26:49.430218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8ccf5-546c-4d27-80b4-17e65cc499b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:26:49.448479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-91e8ccf5-546c-4d27-80b4-17e65cc499b0 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:26:49.450513 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-91e8ccf5-546c-4d27-80b4-17e65cc499b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:26:49.509340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 758/1508] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:26:49.519649 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb33de4-c4ab-4431-bb32-d17453abc4a6 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:26:49.521087 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4eb33de4-c4ab-4431-bb32-d17453abc4a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:26:49.579173 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb33de4-c4ab-4431-bb32-d17453abc4a6 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:26:49.582701 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789846b-1df9-4c82-a75d-a4ef5560a3f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:26:49.583359 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789846b-1df9-4c82-a75d-a4ef5560a3f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:26:49.583643 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789846b-1df9-4c82-a75d-a4ef5560a3f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:26:49.583923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789846b-1df9-4c82-a75d-a4ef5560a3f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:26:49.593230 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789846b-1df9-4c82-a75d-a4ef5560a3f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:26:49.594702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789846b-1df9-4c82-a75d-a4ef5560a3f9 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:26:49.604129 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 751/1509] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:26:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:26:50.063759 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8ccf5-546c-4d27-80b4-17e65cc499b0 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:26:50.065236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-91e8ccf5-546c-4d27-80b4-17e65cc499b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:26:50.085228 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8ccf5-546c-4d27-80b4-17e65cc499b0 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:26:50.092254 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e4566-2bb2-4755-bbeb-d1b54d5a981c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:26:50.093662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e4566-2bb2-4755-bbeb-d1b54d5a981c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:26:50.094218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e4566-2bb2-4755-bbeb-d1b54d5a981c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:26:50.094349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e4566-2bb2-4755-bbeb-d1b54d5a981c admin admin] PATH_INFO: `/v3/projects/d290af33209043d2a319c472d8e3ed4a` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:26:50.095639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e4566-2bb2-4755-bbeb-d1b54d5a981c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d290af33209043d2a319c472d8e3ed4a)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:26:50.096688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: WARNING py.warnings [None req-c47e4566-2bb2-4755-bbeb-d1b54d5a981c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 06:26:50.096688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: warnings.warn(msg) May 19 06:26:50.096688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: May 19 06:26:50.098077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e4566-2bb2-4755-bbeb-d1b54d5a981c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:26:50.163948 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c47e4566-2bb2-4755-bbeb-d1b54d5a981c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd290af33209043d2a319c472d8e3ed4a', 'request_id': 'req-c47e4566-2bb2-4755-bbeb-d1b54d5a981c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:26:50.174532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c47e4566-2bb2-4755-bbeb-d1b54d5a981c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd290af33209043d2a319c472d8e3ed4a', 'request_id': 'req-c47e4566-2bb2-4755-bbeb-d1b54d5a981c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:26:50.180627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c47e4566-2bb2-4755-bbeb-d1b54d5a981c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd290af33209043d2a319c472d8e3ed4a', 'request_id': 'req-c47e4566-2bb2-4755-bbeb-d1b54d5a981c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:26:50.199344 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c47e4566-2bb2-4755-bbeb-d1b54d5a981c admin admin] The token cache is being invalidate because project d290af33209043d2a319c472d8e3ed4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:26:50.199656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c47e4566-2bb2-4755-bbeb-d1b54d5a981c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:26:50.200415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c47e4566-2bb2-4755-bbeb-d1b54d5a981c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:26:50.202118 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 759/1510] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:26:50 2022] DELETE /identity/v3/projects/d290af33209043d2a319c472d8e3ed4a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:26:59.600008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=29 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:27:00.196163 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:28:29.000827 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b63bda-2165-4d4c-93fb-747826571be0 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:29.001446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b63bda-2165-4d4c-93fb-747826571be0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:29.001553 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b63bda-2165-4d4c-93fb-747826571be0 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:29.002893 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 06:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:28:29.007777 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ce6b2-3206-43f4-9a63-2ca5ec6419ce None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:29.008234 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ce6b2-3206-43f4-9a63-2ca5ec6419ce None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:29.008341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ce6b2-3206-43f4-9a63-2ca5ec6419ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:29.026148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-6e9ce6b2-3206-43f4-9a63-2ca5ec6419ce None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:28:29.027866 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6e9ce6b2-3206-43f4-9a63-2ca5ec6419ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:28:29.090082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 06:28:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:28:29.267128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b63bda-2165-4d4c-93fb-747826571be0 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:28:29.268803 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c1b63bda-2165-4d4c-93fb-747826571be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:28:29.346173 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b63bda-2165-4d4c-93fb-747826571be0 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:28:29.351734 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b79f3-ab4f-4e78-a217-4b9d2aff7e73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:28:29.352379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b79f3-ab4f-4e78-a217-4b9d2aff7e73 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:29.352663 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b79f3-ab4f-4e78-a217-4b9d2aff7e73 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:29.352939 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b79f3-ab4f-4e78-a217-4b9d2aff7e73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:29.361757 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b79f3-ab4f-4e78-a217-4b9d2aff7e73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:28:29.363376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b79f3-ab4f-4e78-a217-4b9d2aff7e73 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:28:29.373054 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 753/1513] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:28:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:28:30.166576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff7d0d-60b8-4c63-9b20-2587e95dfe3a None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:30.167635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff7d0d-60b8-4c63-9b20-2587e95dfe3a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:30.167749 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff7d0d-60b8-4c63-9b20-2587e95dfe3a None None] PATH_INFO: `/v3` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:30.169009 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 761/1514] 10.222.0.59 () {54 vars in 925 bytes} [Thu May 19 06:28:30 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:28:30.175112 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100794c-8328-4232-85ce-731a0391c7e3 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:30.175565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100794c-8328-4232-85ce-731a0391c7e3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:30.175683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100794c-8328-4232-85ce-731a0391c7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:30.195664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d100794c-8328-4232-85ce-731a0391c7e3 None None] MFA Rules not processed for user `1b76876d821e4bbbb8d5cad5bd26f559`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:28:30.196907 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d100794c-8328-4232-85ce-731a0391c7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:28:30.279131 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 754/1515] 10.222.0.59 () {58 vars in 1015 bytes} [Thu May 19 06:28:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:28:30.315068 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc2017-508b-43aa-b3c2-0e85b0d3d44a None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:30.315358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc2017-508b-43aa-b3c2-0e85b0d3d44a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:30.315638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc2017-508b-43aa-b3c2-0e85b0d3d44a None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:30.317006 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 762/1516] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 06:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:28:30.322084 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee92e88-cef1-4af0-b014-8ed4bd0db4e5 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:30.322407 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee92e88-cef1-4af0-b014-8ed4bd0db4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:30.322661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee92e88-cef1-4af0-b014-8ed4bd0db4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:30.342186 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8ee92e88-cef1-4af0-b014-8ed4bd0db4e5 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:28:30.343386 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8ee92e88-cef1-4af0-b014-8ed4bd0db4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:28:30.351184 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 755/1517] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 06:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:28:30.502357 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc2017-508b-43aa-b3c2-0e85b0d3d44a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:28:30.517264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc2017-508b-43aa-b3c2-0e85b0d3d44a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:28:30.522645 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28980e7d-9889-484e-adcb-9f8a2e5669b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:28:30.523284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28980e7d-9889-484e-adcb-9f8a2e5669b7 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:30.523574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28980e7d-9889-484e-adcb-9f8a2e5669b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:30.523852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28980e7d-9889-484e-adcb-9f8a2e5669b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:30.532361 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28980e7d-9889-484e-adcb-9f8a2e5669b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:28:30.533966 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28980e7d-9889-484e-adcb-9f8a2e5669b7 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:28:30.543494 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 763/1518] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:28:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:28:34.111828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91523fa9-b435-41b4-8513-55c2be9f568d None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:34.112413 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91523fa9-b435-41b4-8513-55c2be9f568d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:34.112413 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91523fa9-b435-41b4-8513-55c2be9f568d None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:34.113847 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 756/1519] 10.222.0.65 () {54 vars in 960 bytes} [Thu May 19 06:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:28:34.119128 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63a83b-e551-41a4-97cb-90ef13754907 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:34.119559 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63a83b-e551-41a4-97cb-90ef13754907 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:34.119693 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63a83b-e551-41a4-97cb-90ef13754907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:34.138518 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9b63a83b-e551-41a4-97cb-90ef13754907 None None] MFA Rules not processed for user `1b76876d821e4bbbb8d5cad5bd26f559`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:28:34.139744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9b63a83b-e551-41a4-97cb-90ef13754907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:28:34.148289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 764/1520] 10.222.0.65 () {58 vars in 1057 bytes} [Thu May 19 06:28:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:28:34.157239 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91523fa9-b435-41b4-8513-55c2be9f568d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:28:34.170625 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91523fa9-b435-41b4-8513-55c2be9f568d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:28:34.175465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911d9f0-0109-426d-82c3-83bca6484705 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b76876d821e4bbbb8d5cad5bd26f559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:28:34.176094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911d9f0-0109-426d-82c3-83bca6484705 service placement] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:34.176375 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911d9f0-0109-426d-82c3-83bca6484705 service placement] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:34.176649 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911d9f0-0109-426d-82c3-83bca6484705 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:34.185617 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911d9f0-0109-426d-82c3-83bca6484705 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:28:34.188261 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911d9f0-0109-426d-82c3-83bca6484705 service placement] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:28:34.199166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 757/1521] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:28:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:28:41.845485 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e04b8-3fc9-49fd-9b9c-2f33ecff6cfc None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:41.846094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e04b8-3fc9-49fd-9b9c-2f33ecff6cfc None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:41.846094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e04b8-3fc9-49fd-9b9c-2f33ecff6cfc None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:41.847412 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 06:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:28:41.852378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e138f5-dbb5-4657-851c-d7feadcccd1e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:41.852811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e138f5-dbb5-4657-851c-d7feadcccd1e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:41.852955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e138f5-dbb5-4657-851c-d7feadcccd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:41.872259 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-34e138f5-dbb5-4657-851c-d7feadcccd1e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:28:41.873523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-34e138f5-dbb5-4657-851c-d7feadcccd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:28:41.881456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 06:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:28:42.037957 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e04b8-3fc9-49fd-9b9c-2f33ecff6cfc None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:28:42.055021 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e04b8-3fc9-49fd-9b9c-2f33ecff6cfc None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:28:42.060266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d025c-71e4-43ee-937b-9678d52ab410 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:28:42.060919 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d025c-71e4-43ee-937b-9678d52ab410 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:42.061207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d025c-71e4-43ee-937b-9678d52ab410 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:42.061519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d025c-71e4-43ee-937b-9678d52ab410 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:42.070286 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d025c-71e4-43ee-937b-9678d52ab410 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:28:42.071876 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d025c-71e4-43ee-937b-9678d52ab410 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:28:42.082422 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 766/1524] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:28:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:28:42.989208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252fb0-fe0f-4a3a-8201-a8f051e3a3db None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:42.989822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252fb0-fe0f-4a3a-8201-a8f051e3a3db None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:42.989822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252fb0-fe0f-4a3a-8201-a8f051e3a3db None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:42.991163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 06:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:28:42.996883 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba6193-0e8b-45fe-917b-856f2b894016 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:42.997482 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba6193-0e8b-45fe-917b-856f2b894016 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:42.997482 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba6193-0e8b-45fe-917b-856f2b894016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:43.016487 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1eba6193-0e8b-45fe-917b-856f2b894016 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:28:43.017811 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1eba6193-0e8b-45fe-917b-856f2b894016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:28:43.025589 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 06:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:28:43.176339 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2252fb0-fe0f-4a3a-8201-a8f051e3a3db None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:28:43.191087 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2252fb0-fe0f-4a3a-8201-a8f051e3a3db None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:28:43.196519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c0660-8c60-4bd7-88a1-4535605a345c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:28:43.197263 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c0660-8c60-4bd7-88a1-4535605a345c service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:43.197678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c0660-8c60-4bd7-88a1-4535605a345c service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:43.198295 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c0660-8c60-4bd7-88a1-4535605a345c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:43.207079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c0660-8c60-4bd7-88a1-4535605a345c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:28:43.208672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c0660-8c60-4bd7-88a1-4535605a345c service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:28:43.218898 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 760/1527] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:28:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:28:47.251047 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b96bc3-984c-4499-87eb-6e43157af1d5 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:47.251614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b96bc3-984c-4499-87eb-6e43157af1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:47.251614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b96bc3-984c-4499-87eb-6e43157af1d5 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:47.253029 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 06:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:28:47.258746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f2163-ac37-4191-8b3f-5f1df181be1e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:47.259238 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f2163-ac37-4191-8b3f-5f1df181be1e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:47.259355 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f2163-ac37-4191-8b3f-5f1df181be1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:47.279791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-7c5f2163-ac37-4191-8b3f-5f1df181be1e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:28:47.281029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7c5f2163-ac37-4191-8b3f-5f1df181be1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:28:47.291424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 06:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:28:47.330545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b96bc3-984c-4499-87eb-6e43157af1d5 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:28:47.332780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-01b96bc3-984c-4499-87eb-6e43157af1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:28:47.412217 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b96bc3-984c-4499-87eb-6e43157af1d5 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:28:47.416107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3570af4-5c72-4b3b-8774-dcdb09208763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:28:47.417548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3570af4-5c72-4b3b-8774-dcdb09208763 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:47.417847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3570af4-5c72-4b3b-8774-dcdb09208763 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:47.418128 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3570af4-5c72-4b3b-8774-dcdb09208763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:47.430438 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3570af4-5c72-4b3b-8774-dcdb09208763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:28:47.432914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3570af4-5c72-4b3b-8774-dcdb09208763 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:28:47.444567 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 769/1530] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:28:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:28:49.839508 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d618b-bd41-4ef9-a2ad-3bf5aba1c4a3 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:49.840144 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d618b-bd41-4ef9-a2ad-3bf5aba1c4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:49.840144 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d618b-bd41-4ef9-a2ad-3bf5aba1c4a3 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:49.841542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 06:28:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:28:49.847022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5873b-6655-492a-995a-99f6427bebb1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:49.847498 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5873b-6655-492a-995a-99f6427bebb1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:49.847629 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5873b-6655-492a-995a-99f6427bebb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:49.866119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3ea5873b-6655-492a-995a-99f6427bebb1 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:28:49.867310 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3ea5873b-6655-492a-995a-99f6427bebb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:28:49.877682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 06:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:28:49.919185 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d618b-bd41-4ef9-a2ad-3bf5aba1c4a3 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:28:49.921047 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-723d618b-bd41-4ef9-a2ad-3bf5aba1c4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:28:49.928770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=85 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:28:49.936297 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d618b-bd41-4ef9-a2ad-3bf5aba1c4a3 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:28:49.939884 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0737ad8-6043-45bb-87b5-42c68b915996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:28:49.940529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0737ad8-6043-45bb-87b5-42c68b915996 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:49.940816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0737ad8-6043-45bb-87b5-42c68b915996 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:49.941097 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0737ad8-6043-45bb-87b5-42c68b915996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:49.951270 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0737ad8-6043-45bb-87b5-42c68b915996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:28:49.952725 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0737ad8-6043-45bb-87b5-42c68b915996 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:28:49.962299 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 763/1533] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:28:53.652910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedacf10-e542-45eb-b6fb-80cfce2fcce7 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:53.653516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedacf10-e542-45eb-b6fb-80cfce2fcce7 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:53.653516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedacf10-e542-45eb-b6fb-80cfce2fcce7 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:53.654925 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 06:28:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:28:53.660486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40728f-8951-4398-8e88-da58b940c477 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:53.660934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40728f-8951-4398-8e88-da58b940c477 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:53.661045 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40728f-8951-4398-8e88-da58b940c477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:53.678698 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3c40728f-8951-4398-8e88-da58b940c477 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:28:53.679880 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3c40728f-8951-4398-8e88-da58b940c477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:28:53.688121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 06:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:28:53.964463 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedacf10-e542-45eb-b6fb-80cfce2fcce7 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:28:53.966172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-aedacf10-e542-45eb-b6fb-80cfce2fcce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:28:53.971918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=75 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:28:54.026367 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedacf10-e542-45eb-b6fb-80cfce2fcce7 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:28:54.029981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda3dc4-6f66-4856-a5e4-f79d1b313f00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:28:54.030667 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda3dc4-6f66-4856-a5e4-f79d1b313f00 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:54.030963 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda3dc4-6f66-4856-a5e4-f79d1b313f00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:54.031497 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda3dc4-6f66-4856-a5e4-f79d1b313f00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:54.040728 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda3dc4-6f66-4856-a5e4-f79d1b313f00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:28:54.042268 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda3dc4-6f66-4856-a5e4-f79d1b313f00 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:28:54.051797 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 772/1536] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:28:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:28:56.344965 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a93cf-58c1-4263-92e0-b1236a0034c0 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:56.345488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a93cf-58c1-4263-92e0-b1236a0034c0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:56.345592 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a93cf-58c1-4263-92e0-b1236a0034c0 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:56.347006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 06:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:28:56.352124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fafc27-3bf4-4de3-9969-537b9b844557 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:56.352550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fafc27-3bf4-4de3-9969-537b9b844557 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:56.352715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fafc27-3bf4-4de3-9969-537b9b844557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:56.371764 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-46fafc27-3bf4-4de3-9969-537b9b844557 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:28:56.372958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-46fafc27-3bf4-4de3-9969-537b9b844557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:28:56.380679 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 06:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:28:56.655256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a93cf-58c1-4263-92e0-b1236a0034c0 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:28:56.659492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4f2a93cf-58c1-4263-92e0-b1236a0034c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:28:56.672646 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a93cf-58c1-4263-92e0-b1236a0034c0 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:28:56.676226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4189ea6-2f98-486e-aab2-ad90c45d3e8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:28:56.676860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4189ea6-2f98-486e-aab2-ad90c45d3e8c service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:28:56.677147 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4189ea6-2f98-486e-aab2-ad90c45d3e8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:28:56.677467 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4189ea6-2f98-486e-aab2-ad90c45d3e8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:28:56.687245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4189ea6-2f98-486e-aab2-ad90c45d3e8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:28:56.688670 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4189ea6-2f98-486e-aab2-ad90c45d3e8c service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:28:56.699250 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 766/1539] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:28:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:29:06.366025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=28 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:29:06.694009 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=14 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:29.501006 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13a2fa-6de6-48e7-b78e-d8bccaa50df3 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:29.501649 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13a2fa-6de6-48e7-b78e-d8bccaa50df3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:29.501937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13a2fa-6de6-48e7-b78e-d8bccaa50df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:29.545339 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-fd13a2fa-6de6-48e7-b78e-d8bccaa50df3 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:29.547596 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fd13a2fa-6de6-48e7-b78e-d8bccaa50df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:29.573057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 774/1540] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:29.599251 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1379787-48dc-413a-93cf-2f2805faad37 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:29.599910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1379787-48dc-413a-93cf-2f2805faad37 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:29.601273 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1379787-48dc-413a-93cf-2f2805faad37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:29.671304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c1379787-48dc-413a-93cf-2f2805faad37 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:29.673519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c1379787-48dc-413a-93cf-2f2805faad37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:29.693885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 767/1541] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:29.701952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13a2fa-6de6-48e7-b78e-d8bccaa50df3 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:29.743906 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13a2fa-6de6-48e7-b78e-d8bccaa50df3 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:29.766956 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffd15f-d6b4-4d99-b9dc-f2041def8d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:29.768041 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffd15f-d6b4-4d99-b9dc-f2041def8d53 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:29.768506 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffd15f-d6b4-4d99-b9dc-f2041def8d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:29.768999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffd15f-d6b4-4d99-b9dc-f2041def8d53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:29.770390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffd15f-d6b4-4d99-b9dc-f2041def8d53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:30:29.770929 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffd15f-d6b4-4d99-b9dc-f2041def8d53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:29.775281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffd15f-d6b4-4d99-b9dc-f2041def8d53 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:29.835827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 775/1542] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:30:29.846955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1379787-48dc-413a-93cf-2f2805faad37 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:29.876823 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358255d-7050-4f04-af1a-e8abc88f9494 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:29.877397 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358255d-7050-4f04-af1a-e8abc88f9494 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:29.877909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358255d-7050-4f04-af1a-e8abc88f9494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:29.894227 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1379787-48dc-413a-93cf-2f2805faad37 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:29.947244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2358255d-7050-4f04-af1a-e8abc88f9494 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:29.948691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ea64a-6084-4453-839d-93c535609ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:29.949600 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2358255d-7050-4f04-af1a-e8abc88f9494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:29.949877 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ea64a-6084-4453-839d-93c535609ded admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:29.950386 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ea64a-6084-4453-839d-93c535609ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:29.950866 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ea64a-6084-4453-839d-93c535609ded admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:29.952224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ea64a-6084-4453-839d-93c535609ded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:29.965922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ea64a-6084-4453-839d-93c535609ded admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:29.991835 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 776/1543] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:30:29.995854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509ead3-3766-41cc-9808-22e9e4fb1283 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:29.996362 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509ead3-3766-41cc-9808-22e9e4fb1283 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:29.996860 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509ead3-3766-41cc-9808-22e9e4fb1283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:30.059274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3509ead3-3766-41cc-9808-22e9e4fb1283 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:30.061404 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3509ead3-3766-41cc-9808-22e9e4fb1283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:30.085581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 768/1544] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:30:29 2022] POST /identity/v3/projects => generated 367 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:30.091152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1adb280-b0ac-4a73-b870-e39fa9e3eae0 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:30.091664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1adb280-b0ac-4a73-b870-e39fa9e3eae0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:30.092163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1adb280-b0ac-4a73-b870-e39fa9e3eae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:30.126726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 777/1545] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:30.128428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509ead3-3766-41cc-9808-22e9e4fb1283 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:30.193901 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509ead3-3766-41cc-9808-22e9e4fb1283 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:30.208334 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e1adb280-b0ac-4a73-b870-e39fa9e3eae0 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:30.210463 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e1adb280-b0ac-4a73-b870-e39fa9e3eae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:30.559876 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbdbdb-7c22-47c1-888e-b5fcc03bf0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:30.562120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbdbdb-7c22-47c1-888e-b5fcc03bf0a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:30.562854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbdbdb-7c22-47c1-888e-b5fcc03bf0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:30.563600 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbdbdb-7c22-47c1-888e-b5fcc03bf0a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:30.564039 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 769/1546] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:30.565586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbdbdb-7c22-47c1-888e-b5fcc03bf0a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:30.566703 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478609d1-c26b-466f-bd40-a0a8574da6b4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:30.572511 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478609d1-c26b-466f-bd40-a0a8574da6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:30.572511 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478609d1-c26b-466f-bd40-a0a8574da6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:30.576401 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbdbdb-7c22-47c1-888e-b5fcc03bf0a4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:30.643325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-478609d1-c26b-466f-bd40-a0a8574da6b4 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:30.647953 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-478609d1-c26b-466f-bd40-a0a8574da6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:30.656673 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 778/1547] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:30:30 2022] POST /identity/v3/users => generated 284 bytes in 529 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:30.664040 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d3083-422f-4ab9-81c3-6b3a0ece0de1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:30.664592 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d3083-422f-4ab9-81c3-6b3a0ece0de1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:30.665049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d3083-422f-4ab9-81c3-6b3a0ece0de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:30.673388 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 770/1548] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:30.675139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478609d1-c26b-466f-bd40-a0a8574da6b4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:30.692644 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478609d1-c26b-466f-bd40-a0a8574da6b4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:30.694229 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f35d3083-422f-4ab9-81c3-6b3a0ece0de1 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:30.696328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f35d3083-422f-4ab9-81c3-6b3a0ece0de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:30.701855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c627f4-c365-47e7-abdf-0db3c17161fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:30.703101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c627f4-c365-47e7-abdf-0db3c17161fa admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:30.703610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c627f4-c365-47e7-abdf-0db3c17161fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:30.704114 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c627f4-c365-47e7-abdf-0db3c17161fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:30.705533 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c627f4-c365-47e7-abdf-0db3c17161fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:30:30.706080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c627f4-c365-47e7-abdf-0db3c17161fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:30.712531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 779/1549] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:30.712870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: WARNING py.warnings [None req-c1c627f4-c365-47e7-abdf-0db3c17161fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 06:30:30.712870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: warnings.warn(msg) May 19 06:30:30.712870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: May 19 06:30:30.713333 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1da71c-ea4c-48b5-8dff-41aeff49eb36 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:30.713485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c627f4-c365-47e7-abdf-0db3c17161fa admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:30.713635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1da71c-ea4c-48b5-8dff-41aeff49eb36 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:30.713635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1da71c-ea4c-48b5-8dff-41aeff49eb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:30.737721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 771/1550] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:30:30.739940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4472d-ca70-496f-91b4-1f494fea98d6 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:30.740453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4472d-ca70-496f-91b4-1f494fea98d6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:30.740954 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4472d-ca70-496f-91b4-1f494fea98d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:30.751055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-fd1da71c-ea4c-48b5-8dff-41aeff49eb36 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:30.754050 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fd1da71c-ea4c-48b5-8dff-41aeff49eb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:30.771422 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 780/1551] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:30.773758 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1da71c-ea4c-48b5-8dff-41aeff49eb36 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:30.776853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5ec4472d-ca70-496f-91b4-1f494fea98d6 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:30.779112 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5ec4472d-ca70-496f-91b4-1f494fea98d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:30.798103 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1da71c-ea4c-48b5-8dff-41aeff49eb36 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:30.806387 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 772/1552] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:30.808504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf6d36-ebed-4079-a7ae-25c403efb613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:30.809102 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523855e-938c-4524-9dfe-12abc6df7e7e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:30.809718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523855e-938c-4524-9dfe-12abc6df7e7e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:30.810311 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523855e-938c-4524-9dfe-12abc6df7e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:30.811469 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf6d36-ebed-4079-a7ae-25c403efb613 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:30.812238 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf6d36-ebed-4079-a7ae-25c403efb613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:30.813030 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf6d36-ebed-4079-a7ae-25c403efb613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:30.814986 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf6d36-ebed-4079-a7ae-25c403efb613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:30.818987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf6d36-ebed-4079-a7ae-25c403efb613 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:30.851530 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 781/1553] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:30.854608 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf6d36-ebed-4079-a7ae-25c403efb613 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:30.858920 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d523855e-938c-4524-9dfe-12abc6df7e7e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:30.861565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d523855e-938c-4524-9dfe-12abc6df7e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:30.884424 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf6d36-ebed-4079-a7ae-25c403efb613 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:30.887691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 773/1554] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:30.890336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4690aa-50ba-44fa-84d4-47aefd6d5fcf None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:30.890939 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4690aa-50ba-44fa-84d4-47aefd6d5fcf None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:30.891533 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4690aa-50ba-44fa-84d4-47aefd6d5fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:30.896154 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc8e0c-0ed7-4b89-8d3d-6f476a0897c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:30.898132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc8e0c-0ed7-4b89-8d3d-6f476a0897c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:30.899036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc8e0c-0ed7-4b89-8d3d-6f476a0897c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:30.899841 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc8e0c-0ed7-4b89-8d3d-6f476a0897c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:30.901984 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc8e0c-0ed7-4b89-8d3d-6f476a0897c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:30:30.902806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc8e0c-0ed7-4b89-8d3d-6f476a0897c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:30.906238 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc8e0c-0ed7-4b89-8d3d-6f476a0897c9 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:30.920755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-1d4690aa-50ba-44fa-84d4-47aefd6d5fcf None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:30.922664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1d4690aa-50ba-44fa-84d4-47aefd6d5fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:30.937697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 774/1555] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:30.937697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4690aa-50ba-44fa-84d4-47aefd6d5fcf None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:30.938178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 782/1556] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:30:30.942108 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dbc14-b1fa-4f87-9463-22659eccafcd None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:30.942625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dbc14-b1fa-4f87-9463-22659eccafcd None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:30.943130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dbc14-b1fa-4f87-9463-22659eccafcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:30.944835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4690aa-50ba-44fa-84d4-47aefd6d5fcf None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:30.952225 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b07fa-449e-4d05-a09e-30537e4c3c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:30.952928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b07fa-449e-4d05-a09e-30537e4c3c18 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:30.953212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b07fa-449e-4d05-a09e-30537e4c3c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:30.953535 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b07fa-449e-4d05-a09e-30537e4c3c18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:30.954262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b07fa-449e-4d05-a09e-30537e4c3c18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:30.956223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b07fa-449e-4d05-a09e-30537e4c3c18 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:30.973186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-716dbc14-b1fa-4f87-9463-22659eccafcd None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:30.974798 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-716dbc14-b1fa-4f87-9463-22659eccafcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:30.990178 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 775/1557] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:30:30 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:30.991867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e9a10-9a1b-4747-bd9d-702257a05169 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:30.992217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e9a10-9a1b-4747-bd9d-702257a05169 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:30.992565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e9a10-9a1b-4747-bd9d-702257a05169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:31.004807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 783/1558] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:31.004807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716dbc14-b1fa-4f87-9463-22659eccafcd None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:31.022504 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-2c4e9a10-9a1b-4747-bd9d-702257a05169 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:31.022831 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716dbc14-b1fa-4f87-9463-22659eccafcd None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:31.024537 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2c4e9a10-9a1b-4747-bd9d-702257a05169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:31.132197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479ff7e-e428-4fad-a130-c652e10a6769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:31.132960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479ff7e-e428-4fad-a130-c652e10a6769 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:31.134012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479ff7e-e428-4fad-a130-c652e10a6769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:31.134480 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479ff7e-e428-4fad-a130-c652e10a6769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:31.135457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479ff7e-e428-4fad-a130-c652e10a6769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:31.135630 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 776/1559] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:31.142222 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932f384-8257-4c78-8d70-f5219e083144 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:31.142222 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932f384-8257-4c78-8d70-f5219e083144 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:31.142222 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932f384-8257-4c78-8d70-f5219e083144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:31.142699 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479ff7e-e428-4fad-a130-c652e10a6769 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:31.155772 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 784/1560] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:31.157918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60484c-2377-4661-9511-d072f095a88d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:31.158398 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60484c-2377-4661-9511-d072f095a88d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:31.158932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60484c-2377-4661-9511-d072f095a88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:31.175493 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c932f384-8257-4c78-8d70-f5219e083144 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:31.177344 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c932f384-8257-4c78-8d70-f5219e083144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:31.189780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1b60484c-2377-4661-9511-d072f095a88d None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:31.190189 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 777/1561] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:31.191830 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1b60484c-2377-4661-9511-d072f095a88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:31.195170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932f384-8257-4c78-8d70-f5219e083144 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:31.202300 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 785/1562] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:31.204751 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60484c-2377-4661-9511-d072f095a88d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:31.207761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:31.212676 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:31.214814 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932f384-8257-4c78-8d70-f5219e083144 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:31.218817 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60484c-2377-4661-9511-d072f095a88d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:31.223067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895856e7-61a0-4f19-bdd6-e2cb6c499493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:31.224724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895856e7-61a0-4f19-bdd6-e2cb6c499493 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:31.225182 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895856e7-61a0-4f19-bdd6-e2cb6c499493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:31.225676 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895856e7-61a0-4f19-bdd6-e2cb6c499493 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:31.226718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895856e7-61a0-4f19-bdd6-e2cb6c499493 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:30:31.227200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895856e7-61a0-4f19-bdd6-e2cb6c499493 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:31.227368 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fe46e-d9b7-4dd6-9c8c-baf73163cc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:31.227971 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fe46e-d9b7-4dd6-9c8c-baf73163cc9f admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:31.228264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fe46e-d9b7-4dd6-9c8c-baf73163cc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:31.228550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fe46e-d9b7-4dd6-9c8c-baf73163cc9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:31.229325 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fe46e-d9b7-4dd6-9c8c-baf73163cc9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:31.229498 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895856e7-61a0-4f19-bdd6-e2cb6c499493 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:31.230397 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: WARNING py.warnings [None req-1c2fe46e-d9b7-4dd6-9c8c-baf73163cc9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 06:30:31.230397 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: warnings.warn(msg) May 19 06:30:31.230397 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: May 19 06:30:31.231646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fe46e-d9b7-4dd6-9c8c-baf73163cc9f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:31.252396 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 778/1563] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:30:31.254264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895856e7-61a0-4f19-bdd6-e2cb6c499493 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:31.267995 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 786/1564] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:30:31 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:31.267995 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fe46e-d9b7-4dd6-9c8c-baf73163cc9f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:31.274175 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895856e7-61a0-4f19-bdd6-e2cb6c499493 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:31.348657 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fe46e-d9b7-4dd6-9c8c-baf73163cc9f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:31.353997 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58ab85-af1a-4e93-964e-4eb6e3afe832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:31.355541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58ab85-af1a-4e93-964e-4eb6e3afe832 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:31.355847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58ab85-af1a-4e93-964e-4eb6e3afe832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:31.356249 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58ab85-af1a-4e93-964e-4eb6e3afe832 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:31.357089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58ab85-af1a-4e93-964e-4eb6e3afe832 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:30:31.357491 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58ab85-af1a-4e93-964e-4eb6e3afe832 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:31.360631 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58ab85-af1a-4e93-964e-4eb6e3afe832 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:31.376130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 787/1565] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:30:31.377061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58ab85-af1a-4e93-964e-4eb6e3afe832 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:31.393012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58ab85-af1a-4e93-964e-4eb6e3afe832 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:31.396785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ecf66-9a20-47ee-bc1e-86ef33ad9345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:31.397865 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ecf66-9a20-47ee-bc1e-86ef33ad9345 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:31.398216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ecf66-9a20-47ee-bc1e-86ef33ad9345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:31.399289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad8d13-1acc-4e39-9800-ad43ebe4266f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:31.399513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ecf66-9a20-47ee-bc1e-86ef33ad9345 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:31.400515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ecf66-9a20-47ee-bc1e-86ef33ad9345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:31.401992 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: WARNING py.warnings [None req-7c4ecf66-9a20-47ee-bc1e-86ef33ad9345 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 06:30:31.401992 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: warnings.warn(msg) May 19 06:30:31.401992 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: May 19 06:30:31.403428 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ecf66-9a20-47ee-bc1e-86ef33ad9345 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:31.403707 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad8d13-1acc-4e39-9800-ad43ebe4266f admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:31.404469 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad8d13-1acc-4e39-9800-ad43ebe4266f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:31.405705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad8d13-1acc-4e39-9800-ad43ebe4266f admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:31.407536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad8d13-1acc-4e39-9800-ad43ebe4266f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:30:31.408394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad8d13-1acc-4e39-9800-ad43ebe4266f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:31.412373 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad8d13-1acc-4e39-9800-ad43ebe4266f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:31.439063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 779/1566] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:30:31 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:31.440473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ecf66-9a20-47ee-bc1e-86ef33ad9345 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:31.451718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 788/1567] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:30:31.455142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a1029-b823-44a5-ad0e-7498f714c886 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:31.455969 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a1029-b823-44a5-ad0e-7498f714c886 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:31.456745 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a1029-b823-44a5-ad0e-7498f714c886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:31.465996 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ecf66-9a20-47ee-bc1e-86ef33ad9345 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:31.474492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0062c7-525c-42fd-a809-fcc6f94130f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:31.475588 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0062c7-525c-42fd-a809-fcc6f94130f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:31.475945 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0062c7-525c-42fd-a809-fcc6f94130f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:31.476293 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0062c7-525c-42fd-a809-fcc6f94130f1 admin admin] PATH_INFO: `/v3/projects/dd79d3d9a6a846118b918fae7a1384b3/users/7fb40757980043ac8bf116bbc53f6310/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:31.496618 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3f2a1029-b823-44a5-ad0e-7498f714c886 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:31.499375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3f2a1029-b823-44a5-ad0e-7498f714c886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:31.505901 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0062c7-525c-42fd-a809-fcc6f94130f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd79d3d9a6a846118b918fae7a1384b3, user_id=7fb40757980043ac8bf116bbc53f6310, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:31.508966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0062c7-525c-42fd-a809-fcc6f94130f1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:31.516386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 789/1568] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:31.518191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a1029-b823-44a5-ad0e-7498f714c886 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:31.531709 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 780/1569] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:31 2022] PUT /identity/v3/projects/dd79d3d9a6a846118b918fae7a1384b3/users/7fb40757980043ac8bf116bbc53f6310/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:31.532348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0062c7-525c-42fd-a809-fcc6f94130f1 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:31.552010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a1029-b823-44a5-ad0e-7498f714c886 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:31.647742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0062c7-525c-42fd-a809-fcc6f94130f1 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:31.649700 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f417a64-be99-46ad-9aec-d6e610e83c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:31.650615 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f417a64-be99-46ad-9aec-d6e610e83c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:31.650907 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f417a64-be99-46ad-9aec-d6e610e83c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:31.651194 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f417a64-be99-46ad-9aec-d6e610e83c61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:31.651937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f417a64-be99-46ad-9aec-d6e610e83c61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:30:31.652244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f417a64-be99-46ad-9aec-d6e610e83c61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:31.653947 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f417a64-be99-46ad-9aec-d6e610e83c61 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:31.657533 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d004d-f929-4a98-9082-5916dcbae2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:31.658964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d004d-f929-4a98-9082-5916dcbae2f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:31.659410 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d004d-f929-4a98-9082-5916dcbae2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:31.659910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d004d-f929-4a98-9082-5916dcbae2f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:31.661159 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d004d-f929-4a98-9082-5916dcbae2f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:31.665778 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d004d-f929-4a98-9082-5916dcbae2f3 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:31.672505 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 790/1570] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:30:31.672505 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f417a64-be99-46ad-9aec-d6e610e83c61 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:31.693187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f417a64-be99-46ad-9aec-d6e610e83c61 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:31.699932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41803011-3ea6-4be4-8995-075cd436623c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:31.701078 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41803011-3ea6-4be4-8995-075cd436623c admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:31.701458 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41803011-3ea6-4be4-8995-075cd436623c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:31.701939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41803011-3ea6-4be4-8995-075cd436623c admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:31.704840 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41803011-3ea6-4be4-8995-075cd436623c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:31.708528 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41803011-3ea6-4be4-8995-075cd436623c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:31.719335 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 781/1571] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:30:31 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:31.721034 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d004d-f929-4a98-9082-5916dcbae2f3 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:31.746484 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 791/1572] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:30:31 2022] POST /identity/v3/users => generated 289 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:31.747546 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41803011-3ea6-4be4-8995-075cd436623c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:31.764110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41803011-3ea6-4be4-8995-075cd436623c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:31.809589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:31.824823 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:31.847061 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d004d-f929-4a98-9082-5916dcbae2f3 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:31.855480 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb70965-245e-4b3e-af26-3c24ffae8bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:31.856427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb70965-245e-4b3e-af26-3c24ffae8bcc admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:31.856859 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb70965-245e-4b3e-af26-3c24ffae8bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:31.857391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb70965-245e-4b3e-af26-3c24ffae8bcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:31.858529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb70965-245e-4b3e-af26-3c24ffae8bcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:31.861507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb70965-245e-4b3e-af26-3c24ffae8bcc admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:31.879163 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6a102-05aa-4528-accd-deae589d3b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:31.880375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6a102-05aa-4528-accd-deae589d3b0d admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:31.880889 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6a102-05aa-4528-accd-deae589d3b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:31.881415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6a102-05aa-4528-accd-deae589d3b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:31.882862 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6a102-05aa-4528-accd-deae589d3b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:31.886981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6a102-05aa-4528-accd-deae589d3b0d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:31.908023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 792/1573] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:31.909381 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6a102-05aa-4528-accd-deae589d3b0d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:31.913979 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 782/1574] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:30:31 2022] POST /identity/v3/projects => generated 391 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:31.915492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb70965-245e-4b3e-af26-3c24ffae8bcc admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:31.924135 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6a102-05aa-4528-accd-deae589d3b0d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:32.033428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1822c-bb5c-44b8-a8c0-f3d4acb61562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:32.035958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1822c-bb5c-44b8-a8c0-f3d4acb61562 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:32.036501 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1822c-bb5c-44b8-a8c0-f3d4acb61562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:32.037011 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1822c-bb5c-44b8-a8c0-f3d4acb61562 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:32.038455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1822c-bb5c-44b8-a8c0-f3d4acb61562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:32.042817 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1822c-bb5c-44b8-a8c0-f3d4acb61562 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:32.044487 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb70965-245e-4b3e-af26-3c24ffae8bcc admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:32.053837 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c0a3b-8388-4fe9-be36-637764a6e3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:32.055509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c0a3b-8388-4fe9-be36-637764a6e3dc admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:32.055898 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c0a3b-8388-4fe9-be36-637764a6e3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:32.056286 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c0a3b-8388-4fe9-be36-637764a6e3dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:32.057622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c0a3b-8388-4fe9-be36-637764a6e3dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:30:32.058242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c0a3b-8388-4fe9-be36-637764a6e3dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:32.060654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c0a3b-8388-4fe9-be36-637764a6e3dc admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:32.084086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 783/1575] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:30:32.087245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ea58f-3064-43ae-837e-8c3c33f62d6b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:32.087853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ea58f-3064-43ae-837e-8c3c33f62d6b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:32.088386 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ea58f-3064-43ae-837e-8c3c33f62d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:32.111943 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 793/1576] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:30:31 2022] POST /identity/v3/projects => generated 379 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:32.113877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1822c-bb5c-44b8-a8c0-f3d4acb61562 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:32.137509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5e9ea58f-3064-43ae-837e-8c3c33f62d6b None None] MFA Rules not processed for user `7fb40757980043ac8bf116bbc53f6310`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:32.141305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5e9ea58f-3064-43ae-837e-8c3c33f62d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:32.223958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1822c-bb5c-44b8-a8c0-f3d4acb61562 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:32.231179 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245d0db-3fce-4b8a-9892-a10b7c9e1036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:32.232258 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245d0db-3fce-4b8a-9892-a10b7c9e1036 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:32.232720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245d0db-3fce-4b8a-9892-a10b7c9e1036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:32.233191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245d0db-3fce-4b8a-9892-a10b7c9e1036 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:32.234417 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f245d0db-3fce-4b8a-9892-a10b7c9e1036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:32.237561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f245d0db-3fce-4b8a-9892-a10b7c9e1036 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:32.267990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 784/1577] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:32.269726 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ea58f-3064-43ae-837e-8c3c33f62d6b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:32.282663 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 794/1578] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:30:32 2022] POST /identity/v3/projects => generated 405 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:32.283896 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245d0db-3fce-4b8a-9892-a10b7c9e1036 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:32.302944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ea58f-3064-43ae-837e-8c3c33f62d6b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:32.361020 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:32.403782 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245d0db-3fce-4b8a-9892-a10b7c9e1036 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:32.404779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:32.420843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348dcca-b9e3-4081-9df0-816d2a9a07f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:32.422086 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348dcca-b9e3-4081-9df0-816d2a9a07f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:32.422773 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348dcca-b9e3-4081-9df0-816d2a9a07f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:32.423371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348dcca-b9e3-4081-9df0-816d2a9a07f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:32.424595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348dcca-b9e3-4081-9df0-816d2a9a07f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:32.425216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0326455-f001-4863-b285-5a9a7bc07687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:32.426030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0326455-f001-4863-b285-5a9a7bc07687 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:32.426391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0326455-f001-4863-b285-5a9a7bc07687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:32.426752 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0326455-f001-4863-b285-5a9a7bc07687 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:32.427355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348dcca-b9e3-4081-9df0-816d2a9a07f2 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:32.427573 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0326455-f001-4863-b285-5a9a7bc07687 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:32.429660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0326455-f001-4863-b285-5a9a7bc07687 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:32.445915 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 795/1579] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:32.446930 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348dcca-b9e3-4081-9df0-816d2a9a07f2 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:32.460605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348dcca-b9e3-4081-9df0-816d2a9a07f2 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:32.479698 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 785/1580] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:30:32 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:32.480003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e374047-34e6-452b-95c7-f643cd2575ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:32.480003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e374047-34e6-452b-95c7-f643cd2575ac admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:32.480003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e374047-34e6-452b-95c7-f643cd2575ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:32.480003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e374047-34e6-452b-95c7-f643cd2575ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:32.480534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0326455-f001-4863-b285-5a9a7bc07687 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:32.480649 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e374047-34e6-452b-95c7-f643cd2575ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:32.481676 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e374047-34e6-452b-95c7-f643cd2575ac admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:32.497674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0326455-f001-4863-b285-5a9a7bc07687 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:32.505139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 796/1581] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:32.505499 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed88233-ad92-4a3a-8cc9-f0f4d8a87b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:32.506427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed88233-ad92-4a3a-8cc9-f0f4d8a87b47 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:32.506899 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed88233-ad92-4a3a-8cc9-f0f4d8a87b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:32.507367 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed88233-ad92-4a3a-8cc9-f0f4d8a87b47 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:32.507574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e374047-34e6-452b-95c7-f643cd2575ac admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:32.508820 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed88233-ad92-4a3a-8cc9-f0f4d8a87b47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:32.512872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed88233-ad92-4a3a-8cc9-f0f4d8a87b47 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:32.526183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e374047-34e6-452b-95c7-f643cd2575ac admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:32.535439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ab8ae-ae0d-40ef-8682-4d3b621f4a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:32.536413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ab8ae-ae0d-40ef-8682-4d3b621f4a23 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:32.536874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ab8ae-ae0d-40ef-8682-4d3b621f4a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:32.537063 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ab8ae-ae0d-40ef-8682-4d3b621f4a23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:32.537891 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ab8ae-ae0d-40ef-8682-4d3b621f4a23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:32.541085 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ab8ae-ae0d-40ef-8682-4d3b621f4a23 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:32.549403 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 786/1582] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:30:32 2022] POST /identity/v3/users => generated 296 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:32.550890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed88233-ad92-4a3a-8cc9-f0f4d8a87b47 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:32.565132 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed88233-ad92-4a3a-8cc9-f0f4d8a87b47 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:32.576938 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc73b1-4b36-4349-8d2d-3de172f44d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:32.578012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc73b1-4b36-4349-8d2d-3de172f44d1f admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:32.578473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc73b1-4b36-4349-8d2d-3de172f44d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:32.578997 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc73b1-4b36-4349-8d2d-3de172f44d1f admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:32.580095 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 797/1583] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:30:32 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:32.585251 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc73b1-4b36-4349-8d2d-3de172f44d1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:32.585603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ab8ae-ae0d-40ef-8682-4d3b621f4a23 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:32.585756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc73b1-4b36-4349-8d2d-3de172f44d1f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:32.626036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 787/1584] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:30:32 2022] POST /identity/v3/users => generated 290 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:32.627575 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc73b1-4b36-4349-8d2d-3de172f44d1f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:32.646431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc73b1-4b36-4349-8d2d-3de172f44d1f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:32.688703 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ab8ae-ae0d-40ef-8682-4d3b621f4a23 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:32.698054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d7464-7eaf-4ff8-954f-f85ffea591de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:32.698987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d7464-7eaf-4ff8-954f-f85ffea591de admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:32.699377 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d7464-7eaf-4ff8-954f-f85ffea591de admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:32.699823 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d7464-7eaf-4ff8-954f-f85ffea591de admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:32.700699 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d7464-7eaf-4ff8-954f-f85ffea591de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:32.703147 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d7464-7eaf-4ff8-954f-f85ffea591de admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:32.741435 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 798/1585] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:30:32 2022] POST /identity/v3/projects => generated 367 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:32.742475 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d7464-7eaf-4ff8-954f-f85ffea591de admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:32.744018 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63612db1-bef6-4697-bf07-e012d6edcb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:32.745454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63612db1-bef6-4697-bf07-e012d6edcb77 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:32.746080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63612db1-bef6-4697-bf07-e012d6edcb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:32.746705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63612db1-bef6-4697-bf07-e012d6edcb77 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:32.748107 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63612db1-bef6-4697-bf07-e012d6edcb77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:32.752060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63612db1-bef6-4697-bf07-e012d6edcb77 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:32.788278 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 788/1586] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:30:32 2022] POST /identity/v3/users => generated 303 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:32.789069 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63612db1-bef6-4697-bf07-e012d6edcb77 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:32.809196 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63612db1-bef6-4697-bf07-e012d6edcb77 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:32.861061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d7464-7eaf-4ff8-954f-f85ffea591de admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:32.867412 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f6d0a-2a75-4890-868c-b126733bb703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:32.868315 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f6d0a-2a75-4890-868c-b126733bb703 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:32.868604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f6d0a-2a75-4890-868c-b126733bb703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:32.868882 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f6d0a-2a75-4890-868c-b126733bb703 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:32.869957 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f6d0a-2a75-4890-868c-b126733bb703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:32.871963 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f6d0a-2a75-4890-868c-b126733bb703 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:32.892494 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 799/1587] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:32.893941 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f6d0a-2a75-4890-868c-b126733bb703 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:32.911188 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56cdb3-921d-433f-8735-0356c048559d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:32.912280 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56cdb3-921d-433f-8735-0356c048559d admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:32.912743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56cdb3-921d-433f-8735-0356c048559d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:32.913207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56cdb3-921d-433f-8735-0356c048559d admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:32.914403 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f6d0a-2a75-4890-868c-b126733bb703 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:32.915012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56cdb3-921d-433f-8735-0356c048559d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:32.919175 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56cdb3-921d-433f-8735-0356c048559d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:32.921836 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec258e-5c38-4c63-ac62-965baa30c94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:32.923282 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec258e-5c38-4c63-ac62-965baa30c94c admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:32.923772 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec258e-5c38-4c63-ac62-965baa30c94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:32.924248 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec258e-5c38-4c63-ac62-965baa30c94c admin admin] PATH_INFO: `/v3/projects/95990ba77ff749e0aa91bed9d1608e43/users/e6bc1ead7d8947ada932b4544fe6b180/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:32.939817 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 789/1588] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:32.941357 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56cdb3-921d-433f-8735-0356c048559d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:32.950034 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec258e-5c38-4c63-ac62-965baa30c94c admin admin] RBAC: Authorizing `identity:create_grant(project_id=95990ba77ff749e0aa91bed9d1608e43, user_id=e6bc1ead7d8947ada932b4544fe6b180, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:32.951713 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: WARNING py.warnings [None req-8eec258e-5c38-4c63-ac62-965baa30c94c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 06:30:32.951713 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: warnings.warn(msg) May 19 06:30:32.951713 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: May 19 06:30:32.955251 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec258e-5c38-4c63-ac62-965baa30c94c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:32.956543 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56cdb3-921d-433f-8735-0356c048559d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:32.962536 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644ee4a-d746-432e-be7f-77c11d607616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:32.963240 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644ee4a-d746-432e-be7f-77c11d607616 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:32.963529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644ee4a-d746-432e-be7f-77c11d607616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:32.963791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644ee4a-d746-432e-be7f-77c11d607616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:32.964604 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1644ee4a-d746-432e-be7f-77c11d607616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:32.966327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1644ee4a-d746-432e-be7f-77c11d607616 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:32.969083 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:32.984141 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 790/1590] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:32.986428 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644ee4a-d746-432e-be7f-77c11d607616 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:32.988136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 800/1590] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:32 2022] PUT /identity/v3/projects/95990ba77ff749e0aa91bed9d1608e43/users/e6bc1ead7d8947ada932b4544fe6b180/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:32.988978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec258e-5c38-4c63-ac62-965baa30c94c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:33.002679 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644ee4a-d746-432e-be7f-77c11d607616 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:33.043546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:33.106004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391af8cc-a4a7-4f94-b5f0-627462ac0f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:33.106718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391af8cc-a4a7-4f94-b5f0-627462ac0f5f admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.107007 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391af8cc-a4a7-4f94-b5f0-627462ac0f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.107289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391af8cc-a4a7-4f94-b5f0-627462ac0f5f admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.108010 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391af8cc-a4a7-4f94-b5f0-627462ac0f5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:33.110195 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391af8cc-a4a7-4f94-b5f0-627462ac0f5f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:33.111329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec258e-5c38-4c63-ac62-965baa30c94c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:33.120519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d5312-f4fe-465b-a775-0aaf487db941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:33.123183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d5312-f4fe-465b-a775-0aaf487db941 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.123183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d5312-f4fe-465b-a775-0aaf487db941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.123434 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d5312-f4fe-465b-a775-0aaf487db941 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.125189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d5312-f4fe-465b-a775-0aaf487db941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:33.128613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d5312-f4fe-465b-a775-0aaf487db941 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:33.142462 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 801/1591] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:33.143357 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d5312-f4fe-465b-a775-0aaf487db941 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:33.147213 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 791/1592] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:30:32 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:33.149487 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391af8cc-a4a7-4f94-b5f0-627462ac0f5f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:33.159041 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d5312-f4fe-465b-a775-0aaf487db941 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:33.164748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468df7b-4d2d-472f-8b0d-c4f06c3d81d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:33.165643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468df7b-4d2d-472f-8b0d-c4f06c3d81d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.165927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468df7b-4d2d-472f-8b0d-c4f06c3d81d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.166208 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468df7b-4d2d-472f-8b0d-c4f06c3d81d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.166974 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468df7b-4d2d-472f-8b0d-c4f06c3d81d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:33.168980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468df7b-4d2d-472f-8b0d-c4f06c3d81d2 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:33.170584 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391af8cc-a4a7-4f94-b5f0-627462ac0f5f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:33.181019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76aa68-bfc4-4c1a-9c8d-15ead3d56066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:33.182123 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76aa68-bfc4-4c1a-9c8d-15ead3d56066 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.182886 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76aa68-bfc4-4c1a-9c8d-15ead3d56066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.183457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76aa68-bfc4-4c1a-9c8d-15ead3d56066 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.184121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76aa68-bfc4-4c1a-9c8d-15ead3d56066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:33.187293 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76aa68-bfc4-4c1a-9c8d-15ead3d56066 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:33.194337 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 802/1593] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:30:33 2022] POST /identity/v3/users => generated 290 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:33.195205 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468df7b-4d2d-472f-8b0d-c4f06c3d81d2 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:33.206278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468df7b-4d2d-472f-8b0d-c4f06c3d81d2 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:33.210484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 792/1594] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:33.212497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76aa68-bfc4-4c1a-9c8d-15ead3d56066 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:33.214907 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc43b1-cb28-43fc-9043-f461e0a593fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:33.216215 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc43b1-cb28-43fc-9043-f461e0a593fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.216677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc43b1-cb28-43fc-9043-f461e0a593fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.217111 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc43b1-cb28-43fc-9043-f461e0a593fa admin admin] PATH_INFO: `/v3/projects/1e8f1e4b9cd9432b90cb00a6be4d909e/users/7a3536012d1444a1b9b606c9ec94e5cf/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.232456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76aa68-bfc4-4c1a-9c8d-15ead3d56066 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:33.238936 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc43b1-cb28-43fc-9043-f461e0a593fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8f1e4b9cd9432b90cb00a6be4d909e, user_id=7a3536012d1444a1b9b606c9ec94e5cf, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:33.241539 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ea9e9-c26c-4b5d-9cfd-4ef52f2e4329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:33.242805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ea9e9-c26c-4b5d-9cfd-4ef52f2e4329 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.243287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ea9e9-c26c-4b5d-9cfd-4ef52f2e4329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.243786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ea9e9-c26c-4b5d-9cfd-4ef52f2e4329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.245115 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ea9e9-c26c-4b5d-9cfd-4ef52f2e4329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:33.246070 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc43b1-cb28-43fc-9043-f461e0a593fa admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:33.248163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ea9e9-c26c-4b5d-9cfd-4ef52f2e4329 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:33.267325 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 803/1595] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:33 2022] PUT /identity/v3/projects/1e8f1e4b9cd9432b90cb00a6be4d909e/users/7a3536012d1444a1b9b606c9ec94e5cf/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:33.268903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fa0db-6bd4-49d9-9529-6836c7fdfd3e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.269256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fa0db-6bd4-49d9-9529-6836c7fdfd3e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.269598 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fa0db-6bd4-49d9-9529-6836c7fdfd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.276710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 793/1596] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:33.276710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ea9e9-c26c-4b5d-9cfd-4ef52f2e4329 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:33.289205 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ea9e9-c26c-4b5d-9cfd-4ef52f2e4329 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:33.310722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-875fa0db-6bd4-49d9-9529-6836c7fdfd3e None None] MFA Rules not processed for user `e6bc1ead7d8947ada932b4544fe6b180`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:33.312917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-875fa0db-6bd4-49d9-9529-6836c7fdfd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:33.368145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5af24-c75e-4d55-9ee3-d866b60e6799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:33.369195 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5af24-c75e-4d55-9ee3-d866b60e6799 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.369524 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5af24-c75e-4d55-9ee3-d866b60e6799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.369978 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5af24-c75e-4d55-9ee3-d866b60e6799 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.370558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5af24-c75e-4d55-9ee3-d866b60e6799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:33.372213 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5af24-c75e-4d55-9ee3-d866b60e6799 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:33.385493 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 794/1597] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:33.386233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5af24-c75e-4d55-9ee3-d866b60e6799 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:33.397956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5af24-c75e-4d55-9ee3-d866b60e6799 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:33.407004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e65cc9-724e-47e7-9de9-22d796f904cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:33.408400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e65cc9-724e-47e7-9de9-22d796f904cf admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.408890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e65cc9-724e-47e7-9de9-22d796f904cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.409521 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e65cc9-724e-47e7-9de9-22d796f904cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.410991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e65cc9-724e-47e7-9de9-22d796f904cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:33.413851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e65cc9-724e-47e7-9de9-22d796f904cf admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:33.433400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 795/1598] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:33.433400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e65cc9-724e-47e7-9de9-22d796f904cf admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:33.444469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e65cc9-724e-47e7-9de9-22d796f904cf admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:33.454354 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99b222-b72f-4e53-abde-8e65fa548066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:33.455477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99b222-b72f-4e53-abde-8e65fa548066 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.455951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99b222-b72f-4e53-abde-8e65fa548066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.456345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99b222-b72f-4e53-abde-8e65fa548066 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.457741 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99b222-b72f-4e53-abde-8e65fa548066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:33.460369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99b222-b72f-4e53-abde-8e65fa548066 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:33.463818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 804/1599] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:33.465388 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fa0db-6bd4-49d9-9529-6836c7fdfd3e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:33.486859 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 796/1600] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:33.487810 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fa0db-6bd4-49d9-9529-6836c7fdfd3e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:33.489313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99b222-b72f-4e53-abde-8e65fa548066 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:33.500791 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3bd7b-5ef1-40c5-84d0-33a4e97e7c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:33.502779 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3bd7b-5ef1-40c5-84d0-33a4e97e7c14 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.503436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3bd7b-5ef1-40c5-84d0-33a4e97e7c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.504407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3bd7b-5ef1-40c5-84d0-33a4e97e7c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.506428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3bd7b-5ef1-40c5-84d0-33a4e97e7c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:33.509844 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99b222-b72f-4e53-abde-8e65fa548066 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:33.511794 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3bd7b-5ef1-40c5-84d0-33a4e97e7c14 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:33.521651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea95052-dd65-4672-a7de-9c244cd393c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:33.523159 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea95052-dd65-4672-a7de-9c244cd393c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.524311 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea95052-dd65-4672-a7de-9c244cd393c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.525332 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea95052-dd65-4672-a7de-9c244cd393c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.527323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea95052-dd65-4672-a7de-9c244cd393c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:33.530803 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea95052-dd65-4672-a7de-9c244cd393c5 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:33.541895 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 805/1601] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:33.544635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74b602-e8d9-491a-bb3e-2d8cc08b612d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.545269 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74b602-e8d9-491a-bb3e-2d8cc08b612d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.546040 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74b602-e8d9-491a-bb3e-2d8cc08b612d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.552272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 797/1602] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:33.553781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea95052-dd65-4672-a7de-9c244cd393c5 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:33.568444 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea95052-dd65-4672-a7de-9c244cd393c5 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:33.574462 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090bc58-66db-4d35-af12-1db8da564662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:33.575376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090bc58-66db-4d35-af12-1db8da564662 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.575664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090bc58-66db-4d35-af12-1db8da564662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.575944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090bc58-66db-4d35-af12-1db8da564662 admin admin] PATH_INFO: `/v3/projects/adbcff8654844c22a906a2e91ebca3bb/users/cb2bf8531dfe42b29b0b632b39d49625/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.578418 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-af74b602-e8d9-491a-bb3e-2d8cc08b612d None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:33.581007 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-af74b602-e8d9-491a-bb3e-2d8cc08b612d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:33.606069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 806/1603] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:33.607742 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74b602-e8d9-491a-bb3e-2d8cc08b612d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:33.616302 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090bc58-66db-4d35-af12-1db8da564662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adbcff8654844c22a906a2e91ebca3bb, user_id=cb2bf8531dfe42b29b0b632b39d49625, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:33.619605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090bc58-66db-4d35-af12-1db8da564662 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:33.628734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74b602-e8d9-491a-bb3e-2d8cc08b612d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:33.643110 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 798/1604] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:33 2022] PUT /identity/v3/projects/adbcff8654844c22a906a2e91ebca3bb/users/cb2bf8531dfe42b29b0b632b39d49625/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:33.644452 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090bc58-66db-4d35-af12-1db8da564662 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:33.670369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:33.707226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:33.752789 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a7c95-9350-4494-905d-7c8745277fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:33.755356 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a7c95-9350-4494-905d-7c8745277fea admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.756049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a7c95-9350-4494-905d-7c8745277fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.756515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a7c95-9350-4494-905d-7c8745277fea admin admin] PATH_INFO: `/v3/projects/35cb15f2075c41889b91291b645170a5/users/6a71c3c016564c3b9ab0b6b58d378fda/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.764181 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090bc58-66db-4d35-af12-1db8da564662 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:33.773788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ee48d-93a4-4d33-9796-2d5e0082f85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:33.775757 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ee48d-93a4-4d33-9796-2d5e0082f85b admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.776253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ee48d-93a4-4d33-9796-2d5e0082f85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.776712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ee48d-93a4-4d33-9796-2d5e0082f85b admin admin] PATH_INFO: `/v3/projects/d2d7305d881a4ee29213497d3bab5937/users/184b6909ea6649f3b5eadceef3cca359/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.798535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a7c95-9350-4494-905d-7c8745277fea admin admin] RBAC: Authorizing `identity:create_grant(project_id=35cb15f2075c41889b91291b645170a5, user_id=6a71c3c016564c3b9ab0b6b58d378fda, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:33.807692 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ee48d-93a4-4d33-9796-2d5e0082f85b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2d7305d881a4ee29213497d3bab5937, user_id=184b6909ea6649f3b5eadceef3cca359, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:33.809812 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a7c95-9350-4494-905d-7c8745277fea admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:33.813379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ee48d-93a4-4d33-9796-2d5e0082f85b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:33.833670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 807/1605] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:33 2022] PUT /identity/v3/projects/35cb15f2075c41889b91291b645170a5/users/6a71c3c016564c3b9ab0b6b58d378fda/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:33.834966 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a7c95-9350-4494-905d-7c8745277fea admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:33.846843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 799/1606] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:33 2022] PUT /identity/v3/projects/d2d7305d881a4ee29213497d3bab5937/users/184b6909ea6649f3b5eadceef3cca359/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:33.849267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4edaa3-c7fa-4d60-a3cb-852fe4d58ad8 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.849800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4edaa3-c7fa-4d60-a3cb-852fe4d58ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.850389 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4edaa3-c7fa-4d60-a3cb-852fe4d58ad8 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.853058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 800/1607] 10.222.0.65 () {54 vars in 961 bytes} [Thu May 19 06:30:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:30:33.854887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4edaa3-c7fa-4d60-a3cb-852fe4d58ad8 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:33.952402 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a7c95-9350-4494-905d-7c8745277fea admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:33.959944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66265ae-b126-4401-bd44-cc35b39cb4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:33.961024 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66265ae-b126-4401-bd44-cc35b39cb4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.961592 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66265ae-b126-4401-bd44-cc35b39cb4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.962007 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66265ae-b126-4401-bd44-cc35b39cb4b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.963331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66265ae-b126-4401-bd44-cc35b39cb4b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:33.966215 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66265ae-b126-4401-bd44-cc35b39cb4b3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:33.970321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4edaa3-c7fa-4d60-a3cb-852fe4d58ad8 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:33.982610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212ab54-eec0-4f0e-82cb-ebded437ab05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:33.987094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212ab54-eec0-4f0e-82cb-ebded437ab05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:33.988014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 808/1608] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:33.988170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212ab54-eec0-4f0e-82cb-ebded437ab05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:33.988614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212ab54-eec0-4f0e-82cb-ebded437ab05 admin admin] PATH_INFO: `/v3/projects/6fadf42a41e7451b9e3d9845f8c5cfa7/users/f338f2df615d489c91c23e5e291693d2/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:33.990094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66265ae-b126-4401-bd44-cc35b39cb4b3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:34.011471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66265ae-b126-4401-bd44-cc35b39cb4b3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:34.020489 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb28176-d47b-4f19-8872-2badde76aa55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:34.021871 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb28176-d47b-4f19-8872-2badde76aa55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:34.022346 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb28176-d47b-4f19-8872-2badde76aa55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:34.022871 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb28176-d47b-4f19-8872-2badde76aa55 admin admin] PATH_INFO: `/v3/projects/1f4d6a3a84a14ef794dff4f8555e5cec/users/14d78c4d793640bea982327e5e47a3d0/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:34.028868 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212ab54-eec0-4f0e-82cb-ebded437ab05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fadf42a41e7451b9e3d9845f8c5cfa7, user_id=f338f2df615d489c91c23e5e291693d2, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:34.035156 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212ab54-eec0-4f0e-82cb-ebded437ab05 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:34.060754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb28176-d47b-4f19-8872-2badde76aa55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4d6a3a84a14ef794dff4f8555e5cec, user_id=14d78c4d793640bea982327e5e47a3d0, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:34.064482 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 801/1609] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:33 2022] PUT /identity/v3/projects/6fadf42a41e7451b9e3d9845f8c5cfa7/users/f338f2df615d489c91c23e5e291693d2/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:34.066106 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212ab54-eec0-4f0e-82cb-ebded437ab05 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:34.066911 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb28176-d47b-4f19-8872-2badde76aa55 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:34.093691 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 809/1610] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:33 2022] PUT /identity/v3/projects/1f4d6a3a84a14ef794dff4f8555e5cec/users/14d78c4d793640bea982327e5e47a3d0/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:34.103822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b30934-4586-4e35-ac3a-9a9908c6b00f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:34.104126 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b30934-4586-4e35-ac3a-9a9908c6b00f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:34.104656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b30934-4586-4e35-ac3a-9a9908c6b00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:34.131251 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-53b30934-4586-4e35-ac3a-9a9908c6b00f None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:34.132844 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-53b30934-4586-4e35-ac3a-9a9908c6b00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:34.174803 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212ab54-eec0-4f0e-82cb-ebded437ab05 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:34.212637 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 810/1611] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:34.214010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcff47-9ea0-4cb1-88fb-3952b9c04c78 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:34.214301 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcff47-9ea0-4cb1-88fb-3952b9c04c78 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:34.214586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcff47-9ea0-4cb1-88fb-3952b9c04c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:34.236407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-bdfcff47-9ea0-4cb1-88fb-3952b9c04c78 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:34.237704 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bdfcff47-9ea0-4cb1-88fb-3952b9c04c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:34.246933 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 811/1612] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:34.248290 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6531593-5287-494d-9883-a17a6bc18efa None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:34.248581 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6531593-5287-494d-9883-a17a6bc18efa None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:34.248868 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6531593-5287-494d-9883-a17a6bc18efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:34.255396 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:34.265948 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585709b9-aef0-45d6-8fb7-a6fec2051af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:34.267931 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585709b9-aef0-45d6-8fb7-a6fec2051af8 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:34.268555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585709b9-aef0-45d6-8fb7-a6fec2051af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:34.268987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b6531593-5287-494d-9883-a17a6bc18efa None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:34.269438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585709b9-aef0-45d6-8fb7-a6fec2051af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:34.270252 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b6531593-5287-494d-9883-a17a6bc18efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:34.278243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 812/1613] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:34.279679 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56138e-0e6d-40bb-9faf-37b7c6012804 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:34.279959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56138e-0e6d-40bb-9faf-37b7c6012804 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:34.280209 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56138e-0e6d-40bb-9faf-37b7c6012804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:34.288115 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585709b9-aef0-45d6-8fb7-a6fec2051af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:34.290919 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585709b9-aef0-45d6-8fb7-a6fec2051af8 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:34.297013 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:34.303797 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4c56138e-0e6d-40bb-9faf-37b7c6012804 None None] MFA Rules not processed for user `2a09f4878eac4d60a9a7dcedba15c05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:34.306216 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4c56138e-0e6d-40bb-9faf-37b7c6012804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:34.310781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 802/1614] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:34.312089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585709b9-aef0-45d6-8fb7-a6fec2051af8 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:34.335172 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585709b9-aef0-45d6-8fb7-a6fec2051af8 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:34.347378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7ec3c-828a-4dc5-8487-bcd8f1a033e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:34.349464 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7ec3c-828a-4dc5-8487-bcd8f1a033e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:34.351304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7ec3c-828a-4dc5-8487-bcd8f1a033e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:34.352462 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7ec3c-828a-4dc5-8487-bcd8f1a033e8 admin admin] PATH_INFO: `/v3/projects/d68c640e4ce049e1b051070692d205f7/users/8a6e7c96b7c54bf89e7cc7f819bcc135/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:34.381453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7ec3c-828a-4dc5-8487-bcd8f1a033e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d68c640e4ce049e1b051070692d205f7, user_id=8a6e7c96b7c54bf89e7cc7f819bcc135, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:34.387524 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7ec3c-828a-4dc5-8487-bcd8f1a033e8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:34.415760 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 803/1615] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:34 2022] PUT /identity/v3/projects/d68c640e4ce049e1b051070692d205f7/users/8a6e7c96b7c54bf89e7cc7f819bcc135/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:34.417898 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7ec3c-828a-4dc5-8487-bcd8f1a033e8 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:34.432061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 813/1616] 10.222.0.65 () {58 vars in 1058 bytes} [Thu May 19 06:30:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:34.435944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82539fd-5a79-4d20-a9d6-68613f0d1293 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:34.436904 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82539fd-5a79-4d20-a9d6-68613f0d1293 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:34.437876 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82539fd-5a79-4d20-a9d6-68613f0d1293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:34.495952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e82539fd-5a79-4d20-a9d6-68613f0d1293 None None] MFA Rules not processed for user `14d78c4d793640bea982327e5e47a3d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:34.498599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e82539fd-5a79-4d20-a9d6-68613f0d1293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:34.557021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7ec3c-828a-4dc5-8487-bcd8f1a033e8 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:34.571375 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21191225-a5d7-4416-a2a7-41cecaca8c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:34.572982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21191225-a5d7-4416-a2a7-41cecaca8c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:34.573693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21191225-a5d7-4416-a2a7-41cecaca8c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:34.574063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21191225-a5d7-4416-a2a7-41cecaca8c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:34.575557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21191225-a5d7-4416-a2a7-41cecaca8c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:34.579276 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21191225-a5d7-4416-a2a7-41cecaca8c2a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:34.602530 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 804/1617] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:34.604530 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21191225-a5d7-4416-a2a7-41cecaca8c2a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:34.624278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 814/1618] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:34 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:34.624278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82539fd-5a79-4d20-a9d6-68613f0d1293 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:34.671500 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21191225-a5d7-4416-a2a7-41cecaca8c2a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:34.675861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc92d2-bd23-446f-8b24-c5fb52478e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:34.676738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc92d2-bd23-446f-8b24-c5fb52478e14 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:34.676919 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc92d2-bd23-446f-8b24-c5fb52478e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:34.677099 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc92d2-bd23-446f-8b24-c5fb52478e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:34.690627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc92d2-bd23-446f-8b24-c5fb52478e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:34.692131 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc92d2-bd23-446f-8b24-c5fb52478e14 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:34.703962 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 805/1619] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:34.704684 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc92d2-bd23-446f-8b24-c5fb52478e14 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:34.714239 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc92d2-bd23-446f-8b24-c5fb52478e14 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:34.718473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716eeec-5fca-4169-bd65-956b4315d78a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:34.719405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716eeec-5fca-4169-bd65-956b4315d78a service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:34.719696 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716eeec-5fca-4169-bd65-956b4315d78a service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:34.719977 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716eeec-5fca-4169-bd65-956b4315d78a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:34.731215 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716eeec-5fca-4169-bd65-956b4315d78a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:34.732959 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716eeec-5fca-4169-bd65-956b4315d78a service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:34.735107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82539fd-5a79-4d20-a9d6-68613f0d1293 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:34.744358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75a9df-6609-449c-a559-7e6a4c071db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:34.746272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 806/1620] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:34.746442 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75a9df-6609-449c-a559-7e6a4c071db5 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:34.746442 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75a9df-6609-449c-a559-7e6a4c071db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:34.746825 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75a9df-6609-449c-a559-7e6a4c071db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:34.747031 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f8d84-03f5-48e6-8506-b7f09a26df18 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:34.747281 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f8d84-03f5-48e6-8506-b7f09a26df18 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:34.747565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f8d84-03f5-48e6-8506-b7f09a26df18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:34.769836 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75a9df-6609-449c-a559-7e6a4c071db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:34.773981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75a9df-6609-449c-a559-7e6a4c071db5 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:34.787765 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a46f8d84-03f5-48e6-8506-b7f09a26df18 None None] MFA Rules not processed for user `8a6e7c96b7c54bf89e7cc7f819bcc135`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:34.790262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a46f8d84-03f5-48e6-8506-b7f09a26df18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:34.800815 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 815/1621] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:34.802495 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75a9df-6609-449c-a559-7e6a4c071db5 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:34.850925 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:34.872237 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:34.928375 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 807/1622] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:34.932113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f8d84-03f5-48e6-8506-b7f09a26df18 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:34.939383 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75a9df-6609-449c-a559-7e6a4c071db5 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:34.950034 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb2b03-150b-41df-bfcb-8d2ff35af2fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:34.951313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb2b03-150b-41df-bfcb-8d2ff35af2fd service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:34.951313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb2b03-150b-41df-bfcb-8d2ff35af2fd service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:34.951706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb2b03-150b-41df-bfcb-8d2ff35af2fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:34.982406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f8d84-03f5-48e6-8506-b7f09a26df18 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:35.005258 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94d8bf-55ad-4e0a-9459-8599fa58ccb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:35.007323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94d8bf-55ad-4e0a-9459-8599fa58ccb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:35.007827 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94d8bf-55ad-4e0a-9459-8599fa58ccb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:35.008359 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94d8bf-55ad-4e0a-9459-8599fa58ccb2 admin admin] PATH_INFO: `/v3/projects/6fadf42a41e7451b9e3d9845f8c5cfa7/users/f338f2df615d489c91c23e5e291693d2/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:35.014859 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee94d8bf-55ad-4e0a-9459-8599fa58ccb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fadf42a41e7451b9e3d9845f8c5cfa7, user_id=f338f2df615d489c91c23e5e291693d2, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:35.024444 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee94d8bf-55ad-4e0a-9459-8599fa58ccb2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:35.071706 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 808/1623] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:34 2022] PUT /identity/v3/projects/6fadf42a41e7451b9e3d9845f8c5cfa7/users/f338f2df615d489c91c23e5e291693d2/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:35.074995 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94d8bf-55ad-4e0a-9459-8599fa58ccb2 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:35.122769 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb2b03-150b-41df-bfcb-8d2ff35af2fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:35.125043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb2b03-150b-41df-bfcb-8d2ff35af2fd service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:35.158224 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 816/1624] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:35.161271 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22073f-9ebe-41ac-8697-378551b09e83 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:35.162110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22073f-9ebe-41ac-8697-378551b09e83 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:35.163323 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22073f-9ebe-41ac-8697-378551b09e83 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:35.168256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 817/1625] 10.222.0.65 () {54 vars in 961 bytes} [Thu May 19 06:30:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:30:35.170954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5326fe-b40f-4c30-8cb9-8e46cb417268 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:35.174655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5326fe-b40f-4c30-8cb9-8e46cb417268 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:35.175919 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5326fe-b40f-4c30-8cb9-8e46cb417268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:35.255208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94d8bf-55ad-4e0a-9459-8599fa58ccb2 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:35.260496 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-ca5326fe-b40f-4c30-8cb9-8e46cb417268 None None] MFA Rules not processed for user `f338f2df615d489c91c23e5e291693d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:35.263979 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ca5326fe-b40f-4c30-8cb9-8e46cb417268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:35.271179 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d8472-9ccf-448b-96ad-08a8538ad5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:35.272366 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d8472-9ccf-448b-96ad-08a8538ad5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:35.294456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d8472-9ccf-448b-96ad-08a8538ad5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:35.295180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d8472-9ccf-448b-96ad-08a8538ad5a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:35.296536 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d8472-9ccf-448b-96ad-08a8538ad5a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:35.299445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d8472-9ccf-448b-96ad-08a8538ad5a8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:35.354518 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 809/1626] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:30:35 2022] POST /identity/v3/projects => generated 405 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:35.361532 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10b454-aba6-43d1-8e2b-68e3b9024a73 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:35.363712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10b454-aba6-43d1-8e2b-68e3b9024a73 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:35.365191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10b454-aba6-43d1-8e2b-68e3b9024a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:35.416612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-7e10b454-aba6-43d1-8e2b-68e3b9024a73 None None] MFA Rules not processed for user `2a09f4878eac4d60a9a7dcedba15c05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:35.423955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7e10b454-aba6-43d1-8e2b-68e3b9024a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:35.471982 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 818/1627] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:35 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:35.473568 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5326fe-b40f-4c30-8cb9-8e46cb417268 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:35.499891 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5326fe-b40f-4c30-8cb9-8e46cb417268 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:35.582466 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 810/1628] 10.222.0.65 () {58 vars in 1058 bytes} [Thu May 19 06:30:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:35.584774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10b454-aba6-43d1-8e2b-68e3b9024a73 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:35.652314 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:35.674940 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606960b2-45b8-4a1f-a1a3-af5e75812fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:35.675653 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606960b2-45b8-4a1f-a1a3-af5e75812fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:35.677556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606960b2-45b8-4a1f-a1a3-af5e75812fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:35.677866 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606960b2-45b8-4a1f-a1a3-af5e75812fe5 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:35.678612 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606960b2-45b8-4a1f-a1a3-af5e75812fe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:35.682930 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606960b2-45b8-4a1f-a1a3-af5e75812fe5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:35.699320 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:35.710061 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10b454-aba6-43d1-8e2b-68e3b9024a73 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:35.720081 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863c56e-f9c2-4086-98fa-7517a5e353ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:35.724950 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863c56e-f9c2-4086-98fa-7517a5e353ef service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:35.724950 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863c56e-f9c2-4086-98fa-7517a5e353ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:35.726467 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863c56e-f9c2-4086-98fa-7517a5e353ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:35.730810 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 819/1629] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:30:35 2022] POST /identity/v3/users => generated 309 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:35.733202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606960b2-45b8-4a1f-a1a3-af5e75812fe5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:35.762833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606960b2-45b8-4a1f-a1a3-af5e75812fe5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:35.775962 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedb525-ebbd-4b4a-822f-c2962747bc7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:35.777574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedb525-ebbd-4b4a-822f-c2962747bc7b service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:35.778114 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedb525-ebbd-4b4a-822f-c2962747bc7b service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:35.778620 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedb525-ebbd-4b4a-822f-c2962747bc7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:35.889811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863c56e-f9c2-4086-98fa-7517a5e353ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:35.890675 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863c56e-f9c2-4086-98fa-7517a5e353ef service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:35.893953 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedb525-ebbd-4b4a-822f-c2962747bc7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:35.895055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedb525-ebbd-4b4a-822f-c2962747bc7b service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:35.910768 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 820/1630] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:35.911673 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedb525-ebbd-4b4a-822f-c2962747bc7b service nova] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:35.924140 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedb525-ebbd-4b4a-822f-c2962747bc7b service nova] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:35.927571 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 811/1631] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:35 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:35.931133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933ac5f-6cf2-4167-ae9c-a184620f04db None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:35.931962 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933ac5f-6cf2-4167-ae9c-a184620f04db None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:35.934118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933ac5f-6cf2-4167-ae9c-a184620f04db None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:35.939325 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f15ca3-9e2f-4d20-8d84-299325175be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:35.940408 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f15ca3-9e2f-4d20-8d84-299325175be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:35.940408 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f15ca3-9e2f-4d20-8d84-299325175be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:35.940628 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 812/1632] 10.222.0.65 () {54 vars in 912 bytes} [Thu May 19 06:30:35 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:30:35.940720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f15ca3-9e2f-4d20-8d84-299325175be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:35.941258 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f15ca3-9e2f-4d20-8d84-299325175be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:35.942942 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f15ca3-9e2f-4d20-8d84-299325175be5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:35.950598 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdcd09-385e-4173-a3a5-e63d3c834536 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:35.951152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdcd09-385e-4173-a3a5-e63d3c834536 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:35.953710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdcd09-385e-4173-a3a5-e63d3c834536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:35.972392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 821/1633] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:35.973426 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f15ca3-9e2f-4d20-8d84-299325175be5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:35.998222 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f15ca3-9e2f-4d20-8d84-299325175be5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:36.023679 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58391f1-eab3-40e0-9755-76d21c8027e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:36.024646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58391f1-eab3-40e0-9755-76d21c8027e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:36.024931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58391f1-eab3-40e0-9755-76d21c8027e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:36.025207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58391f1-eab3-40e0-9755-76d21c8027e7 admin admin] PATH_INFO: `/v3/projects/4a20802a2cc846719c516a2603daeb04/users/bbfe0368437844ab93976f31da5f4e5d/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:36.045434 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-96bdcd09-385e-4173-a3a5-e63d3c834536 None None] MFA Rules not processed for user `2a09f4878eac4d60a9a7dcedba15c05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:36.054518 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-96bdcd09-385e-4173-a3a5-e63d3c834536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:36.079412 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 813/1634] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 06:30:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:36.083824 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58391f1-eab3-40e0-9755-76d21c8027e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a20802a2cc846719c516a2603daeb04, user_id=bbfe0368437844ab93976f31da5f4e5d, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:36.106651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdcd09-385e-4173-a3a5-e63d3c834536 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:36.113434 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58391f1-eab3-40e0-9755-76d21c8027e7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:36.123209 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdcd09-385e-4173-a3a5-e63d3c834536 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:36.129856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3594c-76f5-4814-9aa5-f00f7c91eb1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:36.130813 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3594c-76f5-4814-9aa5-f00f7c91eb1c service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:36.131100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3594c-76f5-4814-9aa5-f00f7c91eb1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:36.131381 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3594c-76f5-4814-9aa5-f00f7c91eb1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:36.153241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3594c-76f5-4814-9aa5-f00f7c91eb1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:36.158905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3594c-76f5-4814-9aa5-f00f7c91eb1c service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:36.165559 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 822/1635] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:35 2022] PUT /identity/v3/projects/4a20802a2cc846719c516a2603daeb04/users/bbfe0368437844ab93976f31da5f4e5d/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:36.167745 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58391f1-eab3-40e0-9755-76d21c8027e7 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:36.190470 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 814/1636] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:36.411744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58391f1-eab3-40e0-9755-76d21c8027e7 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:36.420406 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009f9c4-3f5e-4903-a10d-b3eb5d933707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:36.421121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009f9c4-3f5e-4903-a10d-b3eb5d933707 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:36.421466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009f9c4-3f5e-4903-a10d-b3eb5d933707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:36.421752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009f9c4-3f5e-4903-a10d-b3eb5d933707 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:36.422508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009f9c4-3f5e-4903-a10d-b3eb5d933707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:36.424169 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009f9c4-3f5e-4903-a10d-b3eb5d933707 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:36.455931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 823/1637] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:36.463485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3594c-76f5-4814-9aa5-f00f7c91eb1c service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:36.480901 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3594c-76f5-4814-9aa5-f00f7c91eb1c service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:36.501028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb6a43-6e3d-4157-976a-be491d539a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:36.507028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb6a43-6e3d-4157-976a-be491d539a6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:36.511718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb6a43-6e3d-4157-976a-be491d539a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:36.513907 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb6a43-6e3d-4157-976a-be491d539a6e admin admin] PATH_INFO: `/v3/projects/4a20802a2cc846719c516a2603daeb04/users/bbfe0368437844ab93976f31da5f4e5d/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:36.537116 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb6a43-6e3d-4157-976a-be491d539a6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a20802a2cc846719c516a2603daeb04, user_id=bbfe0368437844ab93976f31da5f4e5d, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:36.544966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb6a43-6e3d-4157-976a-be491d539a6e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:36.603400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 815/1638] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:36 2022] PUT /identity/v3/projects/4a20802a2cc846719c516a2603daeb04/users/bbfe0368437844ab93976f31da5f4e5d/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:36.629741 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635d59d-3717-4922-b35a-c6ee001ba83d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:36.630154 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635d59d-3717-4922-b35a-c6ee001ba83d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:36.630598 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635d59d-3717-4922-b35a-c6ee001ba83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:36.725374 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cd4f8-2b20-46b0-abf4-26c8bcd79cfc None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:36.725374 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cd4f8-2b20-46b0-abf4-26c8bcd79cfc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:36.725374 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cd4f8-2b20-46b0-abf4-26c8bcd79cfc None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:36.727444 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 816/1639] 10.222.0.65 () {54 vars in 912 bytes} [Thu May 19 06:30:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 06:30:36.742878 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed064ea3-c12e-4bf1-800e-f2621ba056b0 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:36.745380 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed064ea3-c12e-4bf1-800e-f2621ba056b0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:36.745929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed064ea3-c12e-4bf1-800e-f2621ba056b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:36.777041 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0635d59d-3717-4922-b35a-c6ee001ba83d None None] MFA Rules not processed for user `bbfe0368437844ab93976f31da5f4e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:36.778890 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0635d59d-3717-4922-b35a-c6ee001ba83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:36.864980 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ed064ea3-c12e-4bf1-800e-f2621ba056b0 None None] MFA Rules not processed for user `184b6909ea6649f3b5eadceef3cca359`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:36.870933 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ed064ea3-c12e-4bf1-800e-f2621ba056b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:36.895394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:37.042641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 824/1640] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:36 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:37.045727 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288766f0-3ca6-4097-bf30-58e16ac69ea7 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:37.046322 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288766f0-3ca6-4097-bf30-58e16ac69ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:37.046832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288766f0-3ca6-4097-bf30-58e16ac69ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:37.096014 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 817/1641] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:37.099731 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-288766f0-3ca6-4097-bf30-58e16ac69ea7 None None] MFA Rules not processed for user `2a09f4878eac4d60a9a7dcedba15c05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:37.102664 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-288766f0-3ca6-4097-bf30-58e16ac69ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:37.111654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed064ea3-c12e-4bf1-800e-f2621ba056b0 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:37.212567 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed064ea3-c12e-4bf1-800e-f2621ba056b0 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:37.216582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e1954-21ae-42ee-82f7-c9ba9ca03ee3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:37.217251 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e1954-21ae-42ee-82f7-c9ba9ca03ee3 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:37.218144 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e1954-21ae-42ee-82f7-c9ba9ca03ee3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:37.218144 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e1954-21ae-42ee-82f7-c9ba9ca03ee3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:37.228293 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:37.234543 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e1954-21ae-42ee-82f7-c9ba9ca03ee3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:37.241206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e1954-21ae-42ee-82f7-c9ba9ca03ee3 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:37.266297 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 818/1642] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:37 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:37.267823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d170e47-2e71-4968-b969-91166a9e364f None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:37.268164 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d170e47-2e71-4968-b969-91166a9e364f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:37.268500 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d170e47-2e71-4968-b969-91166a9e364f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:37.295883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3d170e47-2e71-4968-b969-91166a9e364f None None] MFA Rules not processed for user `184b6909ea6649f3b5eadceef3cca359`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:37.299364 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3d170e47-2e71-4968-b969-91166a9e364f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:37.335152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 819/1643] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:37.340881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 825/1644] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 06:30:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:37.413653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d170e47-2e71-4968-b969-91166a9e364f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:37.428008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d170e47-2e71-4968-b969-91166a9e364f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:37.435831 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5b409-4976-4b61-9a37-cf1d06f541c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:37.436951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5b409-4976-4b61-9a37-cf1d06f541c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:37.437521 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5b409-4976-4b61-9a37-cf1d06f541c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:37.438021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5b409-4976-4b61-9a37-cf1d06f541c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:37.460098 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5b409-4976-4b61-9a37-cf1d06f541c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:37.462384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5b409-4976-4b61-9a37-cf1d06f541c2 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:37.487926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 820/1645] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:37.625497 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288766f0-3ca6-4097-bf30-58e16ac69ea7 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:37.658464 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288766f0-3ca6-4097-bf30-58e16ac69ea7 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:37.676769 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89763a-ad5f-4b3e-b517-d49410f0f66e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:37.678806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89763a-ad5f-4b3e-b517-d49410f0f66e service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:37.679526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89763a-ad5f-4b3e-b517-d49410f0f66e service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:37.681515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89763a-ad5f-4b3e-b517-d49410f0f66e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:37.701013 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89763a-ad5f-4b3e-b517-d49410f0f66e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:37.705685 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89763a-ad5f-4b3e-b517-d49410f0f66e service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:37.746430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 826/1646] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:38.125496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5b409-4976-4b61-9a37-cf1d06f541c2 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:38.284196 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5b409-4976-4b61-9a37-cf1d06f541c2 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:38.297244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c239f-7a08-40d4-aeab-73fa1964f924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:38.297244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c239f-7a08-40d4-aeab-73fa1964f924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:38.297244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c239f-7a08-40d4-aeab-73fa1964f924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:38.298161 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c239f-7a08-40d4-aeab-73fa1964f924 admin admin] PATH_INFO: `/v3/users/14d78c4d793640bea982327e5e47a3d0` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:38.302428 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c239f-7a08-40d4-aeab-73fa1964f924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14d78c4d793640bea982327e5e47a3d0)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:38.307335 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c239f-7a08-40d4-aeab-73fa1964f924 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:38.404587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89763a-ad5f-4b3e-b517-d49410f0f66e service nova] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:38.443868 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89763a-ad5f-4b3e-b517-d49410f0f66e service nova] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:38.448875 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-826c239f-7a08-40d4-aeab-73fa1964f924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14d78c4d793640bea982327e5e47a3d0'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:38.452256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-826c239f-7a08-40d4-aeab-73fa1964f924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14d78c4d793640bea982327e5e47a3d0'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:38.469845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f12aed-491a-4577-8c88-448d5b13a2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:38.473236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f12aed-491a-4577-8c88-448d5b13a2bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:38.476703 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f12aed-491a-4577-8c88-448d5b13a2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:38.482329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f12aed-491a-4577-8c88-448d5b13a2bf admin admin] PATH_INFO: `/v3/users/8a6e7c96b7c54bf89e7cc7f819bcc135` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:38.487269 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f12aed-491a-4577-8c88-448d5b13a2bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a6e7c96b7c54bf89e7cc7f819bcc135)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:38.490424 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: WARNING py.warnings [None req-60f12aed-491a-4577-8c88-448d5b13a2bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 06:30:38.490424 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: warnings.warn(msg) May 19 06:30:38.490424 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: May 19 06:30:38.492988 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-826c239f-7a08-40d4-aeab-73fa1964f924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14d78c4d793640bea982327e5e47a3d0'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:38.494101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-826c239f-7a08-40d4-aeab-73fa1964f924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14d78c4d793640bea982327e5e47a3d0'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:38.498997 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f12aed-491a-4577-8c88-448d5b13a2bf admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:38.515551 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-826c239f-7a08-40d4-aeab-73fa1964f924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14d78c4d793640bea982327e5e47a3d0'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:38.550515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 821/1647] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:30:38 2022] DELETE /identity/v3/users/14d78c4d793640bea982327e5e47a3d0 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:30:38.557909 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a480a6c-bcea-467e-9dac-b100f99e045b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:38.558203 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a480a6c-bcea-467e-9dac-b100f99e045b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:38.558506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a480a6c-bcea-467e-9dac-b100f99e045b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:38.582041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-7a480a6c-bcea-467e-9dac-b100f99e045b None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:38.584109 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7a480a6c-bcea-467e-9dac-b100f99e045b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:38.712075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-60f12aed-491a-4577-8c88-448d5b13a2bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a6e7c96b7c54bf89e7cc7f819bcc135'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:38.786446 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-60f12aed-491a-4577-8c88-448d5b13a2bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a6e7c96b7c54bf89e7cc7f819bcc135'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:38.799751 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-60f12aed-491a-4577-8c88-448d5b13a2bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a6e7c96b7c54bf89e7cc7f819bcc135'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:38.801103 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-60f12aed-491a-4577-8c88-448d5b13a2bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a6e7c96b7c54bf89e7cc7f819bcc135'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:38.802809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-60f12aed-491a-4577-8c88-448d5b13a2bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a6e7c96b7c54bf89e7cc7f819bcc135'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:38.815175 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:38.849694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 822/1648] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:30:38.879983 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 827/1649] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:30:38 2022] DELETE /identity/v3/users/8a6e7c96b7c54bf89e7cc7f819bcc135 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:38.882898 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f678d0-354d-4bb6-a267-1a84ef3a5ed0 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:38.884665 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f678d0-354d-4bb6-a267-1a84ef3a5ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:38.885087 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f678d0-354d-4bb6-a267-1a84ef3a5ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:38.928404 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-72f678d0-354d-4bb6-a267-1a84ef3a5ed0 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:38.931381 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-72f678d0-354d-4bb6-a267-1a84ef3a5ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:38.934826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f12aed-491a-4577-8c88-448d5b13a2bf admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:38.937479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-60f12aed-491a-4577-8c88-448d5b13a2bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:39.051587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:39.171180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f12aed-491a-4577-8c88-448d5b13a2bf admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:39.171180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d02a0-ad06-4b8c-a7a5-9878685172ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:39.171180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d02a0-ad06-4b8c-a7a5-9878685172ed service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:39.171180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d02a0-ad06-4b8c-a7a5-9878685172ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:39.171493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d02a0-ad06-4b8c-a7a5-9878685172ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:39.171599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 823/1650] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:30:39.171599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5f416-ffa5-41fd-91ab-89b5b8bed39c None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:39.171599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5f416-ffa5-41fd-91ab-89b5b8bed39c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:39.171599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5f416-ffa5-41fd-91ab-89b5b8bed39c None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:39.171599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 824/1651] 10.222.0.65 () {54 vars in 912 bytes} [Thu May 19 06:30:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:30:39.201540 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d02a0-ad06-4b8c-a7a5-9878685172ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:39.202187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79af830-3bcf-4038-ae31-0978e013b4ab None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:39.203316 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d02a0-ad06-4b8c-a7a5-9878685172ed service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:39.204304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79af830-3bcf-4038-ae31-0978e013b4ab None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:39.205864 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79af830-3bcf-4038-ae31-0978e013b4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:39.235055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 828/1652] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:39.235865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d02a0-ad06-4b8c-a7a5-9878685172ed service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:39.248740 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f53d02a0-ad06-4b8c-a7a5-9878685172ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:39.296810 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d02a0-ad06-4b8c-a7a5-9878685172ed service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:39.303800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c79af830-3bcf-4038-ae31-0978e013b4ab None None] MFA Rules not processed for user `226f4700a516415cbbd272fdfad7af58`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:39.311964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c79af830-3bcf-4038-ae31-0978e013b4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:39.315736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91464c5f-907f-4395-aa42-4b16d0aaff93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:39.316800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91464c5f-907f-4395-aa42-4b16d0aaff93 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:39.317324 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91464c5f-907f-4395-aa42-4b16d0aaff93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:39.317783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91464c5f-907f-4395-aa42-4b16d0aaff93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:39.338473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 825/1653] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 06:30:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:39.376066 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91464c5f-907f-4395-aa42-4b16d0aaff93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:39.378841 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91464c5f-907f-4395-aa42-4b16d0aaff93 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:39.415930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79af830-3bcf-4038-ae31-0978e013b4ab None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:39.436095 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 829/1654] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:39.447432 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79af830-3bcf-4038-ae31-0978e013b4ab None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:39.464653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5494a8c-091e-4753-8c36-5d7b609f9d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:39.465965 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5494a8c-091e-4753-8c36-5d7b609f9d96 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:39.466164 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5494a8c-091e-4753-8c36-5d7b609f9d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:39.466451 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5494a8c-091e-4753-8c36-5d7b609f9d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:39.482038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5494a8c-091e-4753-8c36-5d7b609f9d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:39.484234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5494a8c-091e-4753-8c36-5d7b609f9d96 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:39.500822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 826/1655] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:40.842148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91464c5f-907f-4395-aa42-4b16d0aaff93 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:40.844411 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-91464c5f-907f-4395-aa42-4b16d0aaff93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:40.869625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91464c5f-907f-4395-aa42-4b16d0aaff93 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:40.892826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886abacf-dae7-4b22-9645-484e8434b0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:40.892826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886abacf-dae7-4b22-9645-484e8434b0df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:40.892826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886abacf-dae7-4b22-9645-484e8434b0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:40.895612 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886abacf-dae7-4b22-9645-484e8434b0df admin admin] PATH_INFO: `/v3/users/7fb40757980043ac8bf116bbc53f6310` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:40.900570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886abacf-dae7-4b22-9645-484e8434b0df admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fb40757980043ac8bf116bbc53f6310)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:40.900570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886abacf-dae7-4b22-9645-484e8434b0df admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:40.916932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5494a8c-091e-4753-8c36-5d7b609f9d96 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:40.922603 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c5494a8c-091e-4753-8c36-5d7b609f9d96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:41.000085 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5494a8c-091e-4753-8c36-5d7b609f9d96 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:41.014550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-886abacf-dae7-4b22-9645-484e8434b0df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fb40757980043ac8bf116bbc53f6310'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:41.019034 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4381c2-936c-444f-aebe-fcaf3430a45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:41.021587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4381c2-936c-444f-aebe-fcaf3430a45a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:41.022712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4381c2-936c-444f-aebe-fcaf3430a45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:41.023150 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4381c2-936c-444f-aebe-fcaf3430a45a admin admin] PATH_INFO: `/v3/users/e6bc1ead7d8947ada932b4544fe6b180` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:41.027854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4381c2-936c-444f-aebe-fcaf3430a45a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6bc1ead7d8947ada932b4544fe6b180)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:41.034798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4381c2-936c-444f-aebe-fcaf3430a45a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:41.049984 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-886abacf-dae7-4b22-9645-484e8434b0df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fb40757980043ac8bf116bbc53f6310'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:41.071723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-886abacf-dae7-4b22-9645-484e8434b0df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fb40757980043ac8bf116bbc53f6310'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:41.073343 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-886abacf-dae7-4b22-9645-484e8434b0df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fb40757980043ac8bf116bbc53f6310'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:41.075192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-886abacf-dae7-4b22-9645-484e8434b0df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fb40757980043ac8bf116bbc53f6310'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:41.180655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 830/1656] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:30:40 2022] DELETE /identity/v3/users/7fb40757980043ac8bf116bbc53f6310 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:30:41.183698 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886abacf-dae7-4b22-9645-484e8434b0df admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:41.192438 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-886abacf-dae7-4b22-9645-484e8434b0df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:41.194039 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ea4381c2-936c-444f-aebe-fcaf3430a45a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6bc1ead7d8947ada932b4544fe6b180'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:41.196062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ea4381c2-936c-444f-aebe-fcaf3430a45a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6bc1ead7d8947ada932b4544fe6b180'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:41.233097 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ea4381c2-936c-444f-aebe-fcaf3430a45a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6bc1ead7d8947ada932b4544fe6b180'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:41.235944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ea4381c2-936c-444f-aebe-fcaf3430a45a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6bc1ead7d8947ada932b4544fe6b180'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:41.256673 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ea4381c2-936c-444f-aebe-fcaf3430a45a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6bc1ead7d8947ada932b4544fe6b180'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:41.280958 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 827/1657] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:30:40 2022] DELETE /identity/v3/users/e6bc1ead7d8947ada932b4544fe6b180 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:41.282670 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987f9ae-ff0e-40f2-ab57-af115a6a15dd None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:41.283197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987f9ae-ff0e-40f2-ab57-af115a6a15dd None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:41.283807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987f9ae-ff0e-40f2-ab57-af115a6a15dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:41.329886 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-4987f9ae-ff0e-40f2-ab57-af115a6a15dd None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:41.331525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4987f9ae-ff0e-40f2-ab57-af115a6a15dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:41.344061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886abacf-dae7-4b22-9645-484e8434b0df admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:41.385436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:41.412131 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:41.517888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:41.519010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:41.520349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:41.520712 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd admin admin] PATH_INFO: `/v3/projects/1f4d6a3a84a14ef794dff4f8555e5cec` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:41.525397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 828/1658] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:41.526860 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f4d6a3a84a14ef794dff4f8555e5cec)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:41.530097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:41.539559 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b0f50-c551-4a2a-8b85-d28059669c08 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:41.544240 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b0f50-c551-4a2a-8b85-d28059669c08 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:41.544709 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b0f50-c551-4a2a-8b85-d28059669c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:41.591825 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-273b0f50-c551-4a2a-8b85-d28059669c08 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:41.594527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-273b0f50-c551-4a2a-8b85-d28059669c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:41.615602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 829/1659] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:41.628647 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6abcc-9ade-4569-9ea5-a4f7bc5fa7f8 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:41.629328 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6abcc-9ade-4569-9ea5-a4f7bc5fa7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:41.629471 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6abcc-9ade-4569-9ea5-a4f7bc5fa7f8 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:41.631321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 830/1660] 10.222.0.65 () {54 vars in 960 bytes} [Thu May 19 06:30:41 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:30:41.632154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6abcc-9ade-4569-9ea5-a4f7bc5fa7f8 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:41.636379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-60f6abcc-9ade-4569-9ea5-a4f7bc5fa7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:41.684154 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f4d6a3a84a14ef794dff4f8555e5cec', 'request_id': 'req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:41.702373 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f4d6a3a84a14ef794dff4f8555e5cec', 'request_id': 'req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:41.720641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f4d6a3a84a14ef794dff4f8555e5cec', 'request_id': 'req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:41.763396 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd admin admin] The token cache is being invalidate because project 1f4d6a3a84a14ef794dff4f8555e5cec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:30:41.763961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:41.768005 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:41.771956 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 831/1661] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:30:41 2022] DELETE /identity/v3/projects/1f4d6a3a84a14ef794dff4f8555e5cec => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:41.773025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:41.778803 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:41.895782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6abcc-9ade-4569-9ea5-a4f7bc5fa7f8 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:41.909451 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7b7e1-c8fc-425a-a892-f3984fe65362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:41.911464 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7b7e1-c8fc-425a-a892-f3984fe65362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:41.911971 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7b7e1-c8fc-425a-a892-f3984fe65362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:41.912908 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7b7e1-c8fc-425a-a892-f3984fe65362 admin admin] PATH_INFO: `/v3/projects/d68c640e4ce049e1b051070692d205f7` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:41.930866 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a9cd8-84c1-48b2-8f20-96777ef2adcd admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:41.931758 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7b7e1-c8fc-425a-a892-f3984fe65362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d68c640e4ce049e1b051070692d205f7)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:41.936994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: WARNING py.warnings [None req-d8b7b7e1-c8fc-425a-a892-f3984fe65362 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 06:30:41.936994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: warnings.warn(msg) May 19 06:30:41.936994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: May 19 06:30:41.944718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc40dd-7f7f-4781-842a-77b52add90bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:41.945781 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc40dd-7f7f-4781-842a-77b52add90bc service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:41.946133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc40dd-7f7f-4781-842a-77b52add90bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:41.946484 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc40dd-7f7f-4781-842a-77b52add90bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:41.947423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7b7e1-c8fc-425a-a892-f3984fe65362 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:41.948975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d6bc40dd-7f7f-4781-842a-77b52add90bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:41.976119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc40dd-7f7f-4781-842a-77b52add90bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:41.979385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc40dd-7f7f-4781-842a-77b52add90bc service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:41.996026 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 832/1662] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:41.997378 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc40dd-7f7f-4781-842a-77b52add90bc service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:42.016149 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc40dd-7f7f-4781-842a-77b52add90bc service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:42.024960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050b8a2-8b01-45fc-b1ac-5394bc999e89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:42.025993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050b8a2-8b01-45fc-b1ac-5394bc999e89 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:42.026489 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050b8a2-8b01-45fc-b1ac-5394bc999e89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:42.026960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050b8a2-8b01-45fc-b1ac-5394bc999e89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:42.029880 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5050b8a2-8b01-45fc-b1ac-5394bc999e89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:42.046051 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5050b8a2-8b01-45fc-b1ac-5394bc999e89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:42.048152 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5050b8a2-8b01-45fc-b1ac-5394bc999e89 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:42.068247 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 833/1663] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:42.070860 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33985cc-e325-419b-ad87-d2620b6db3f2 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:42.071464 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33985cc-e325-419b-ad87-d2620b6db3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:42.071970 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33985cc-e325-419b-ad87-d2620b6db3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:42.120654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d33985cc-e325-419b-ad87-d2620b6db3f2 None None] MFA Rules not processed for user `1b76876d821e4bbbb8d5cad5bd26f559`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:42.130655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d33985cc-e325-419b-ad87-d2620b6db3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:42.237829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d8b7b7e1-c8fc-425a-a892-f3984fe65362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd68c640e4ce049e1b051070692d205f7', 'request_id': 'req-d8b7b7e1-c8fc-425a-a892-f3984fe65362'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:42.251473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d8b7b7e1-c8fc-425a-a892-f3984fe65362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd68c640e4ce049e1b051070692d205f7', 'request_id': 'req-d8b7b7e1-c8fc-425a-a892-f3984fe65362'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:42.288797 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d8b7b7e1-c8fc-425a-a892-f3984fe65362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd68c640e4ce049e1b051070692d205f7', 'request_id': 'req-d8b7b7e1-c8fc-425a-a892-f3984fe65362'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:42.346793 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:42.408444 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d8b7b7e1-c8fc-425a-a892-f3984fe65362 admin admin] The token cache is being invalidate because project d68c640e4ce049e1b051070692d205f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:30:42.408945 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d8b7b7e1-c8fc-425a-a892-f3984fe65362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:42.412754 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d8b7b7e1-c8fc-425a-a892-f3984fe65362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:42.421479 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 831/1664] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:30:41 2022] DELETE /identity/v3/projects/d68c640e4ce049e1b051070692d205f7 => generated 0 bytes in 784 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:42.421479 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7b7e1-c8fc-425a-a892-f3984fe65362 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:42.433355 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d8b7b7e1-c8fc-425a-a892-f3984fe65362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:42.454329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 834/1665] 10.222.0.65 () {58 vars in 1057 bytes} [Thu May 19 06:30:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:42.466172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07b883-66ac-47db-b7a2-15e29bae524c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:42.467171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07b883-66ac-47db-b7a2-15e29bae524c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:42.468706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07b883-66ac-47db-b7a2-15e29bae524c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:42.573021 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1e07b883-66ac-47db-b7a2-15e29bae524c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:42.575620 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1e07b883-66ac-47db-b7a2-15e29bae524c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:42.802241 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 835/1666] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:42.806663 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07b883-66ac-47db-b7a2-15e29bae524c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:42.813364 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7b7e1-c8fc-425a-a892-f3984fe65362 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:42.835631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98642b-15ca-4cf5-a955-06052ab1e555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:42.836765 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98642b-15ca-4cf5-a955-06052ab1e555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:42.837166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98642b-15ca-4cf5-a955-06052ab1e555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:42.839661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98642b-15ca-4cf5-a955-06052ab1e555 admin admin] PATH_INFO: `/v3/users/bbfe0368437844ab93976f31da5f4e5d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:42.842210 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98642b-15ca-4cf5-a955-06052ab1e555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbfe0368437844ab93976f31da5f4e5d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:42.848210 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98642b-15ca-4cf5-a955-06052ab1e555 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:42.852117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:43.093710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9e98642b-15ca-4cf5-a955-06052ab1e555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbfe0368437844ab93976f31da5f4e5d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:43.095354 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9e98642b-15ca-4cf5-a955-06052ab1e555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbfe0368437844ab93976f31da5f4e5d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:43.130009 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07b883-66ac-47db-b7a2-15e29bae524c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:43.131899 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9e98642b-15ca-4cf5-a955-06052ab1e555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbfe0368437844ab93976f31da5f4e5d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:43.133386 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9e98642b-15ca-4cf5-a955-06052ab1e555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbfe0368437844ab93976f31da5f4e5d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:43.150637 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9e98642b-15ca-4cf5-a955-06052ab1e555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbfe0368437844ab93976f31da5f4e5d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:43.162191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab1318-c24a-44b7-9c9d-e88940d2f76c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b76876d821e4bbbb8d5cad5bd26f559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:43.165143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab1318-c24a-44b7-9c9d-e88940d2f76c service placement] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:43.168921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab1318-c24a-44b7-9c9d-e88940d2f76c service placement] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:43.169229 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab1318-c24a-44b7-9c9d-e88940d2f76c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:43.177262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-37ab1318-c24a-44b7-9c9d-e88940d2f76c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:43.217136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 832/1667] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:30:42 2022] DELETE /identity/v3/users/bbfe0368437844ab93976f31da5f4e5d => generated 0 bytes in 800 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:43.220191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65d361-56b2-4573-86b4-ddcb183dded9 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:43.222367 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65d361-56b2-4573-86b4-ddcb183dded9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:43.223761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65d361-56b2-4573-86b4-ddcb183dded9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:43.245889 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab1318-c24a-44b7-9c9d-e88940d2f76c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:43.254807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab1318-c24a-44b7-9c9d-e88940d2f76c service placement] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:43.296829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-db65d361-56b2-4573-86b4-ddcb183dded9 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:43.299572 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-db65d361-56b2-4573-86b4-ddcb183dded9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:43.369793 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 836/1668] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:43.721793 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 833/1669] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:43.724366 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab1318-c24a-44b7-9c9d-e88940d2f76c service placement] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:43.766514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab1318-c24a-44b7-9c9d-e88940d2f76c service placement] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:43.810546 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa30ff-3f96-47d8-98a3-8479d8d78da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:43.812093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa30ff-3f96-47d8-98a3-8479d8d78da3 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:43.812814 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa30ff-3f96-47d8-98a3-8479d8d78da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:43.814134 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa30ff-3f96-47d8-98a3-8479d8d78da3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:43.819169 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa30ff-3f96-47d8-98a3-8479d8d78da3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:30:43.821752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa30ff-3f96-47d8-98a3-8479d8d78da3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:43.825360 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa30ff-3f96-47d8-98a3-8479d8d78da3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:43.929389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 837/1670] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:30:43.930993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65d361-56b2-4573-86b4-ddcb183dded9 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:43.965225 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65d361-56b2-4573-86b4-ddcb183dded9 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:44.011419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf458a-70c7-4460-aca7-9257014d08c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:44.013100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf458a-70c7-4460-aca7-9257014d08c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:44.021400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf458a-70c7-4460-aca7-9257014d08c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:44.022282 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf458a-70c7-4460-aca7-9257014d08c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:44.027090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf458a-70c7-4460-aca7-9257014d08c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:44.048445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf458a-70c7-4460-aca7-9257014d08c7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:44.141545 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 834/1671] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:30:43 2022] POST /identity/v3/projects => generated 365 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:44.148677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa30ff-3f96-47d8-98a3-8479d8d78da3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:44.195136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa30ff-3f96-47d8-98a3-8479d8d78da3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:44.589412 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45539a-512e-40c6-9fae-f168b7e565df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:44.590597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45539a-512e-40c6-9fae-f168b7e565df admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:44.591109 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45539a-512e-40c6-9fae-f168b7e565df admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:44.592391 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45539a-512e-40c6-9fae-f168b7e565df admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:44.599746 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45539a-512e-40c6-9fae-f168b7e565df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:44.612337 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45539a-512e-40c6-9fae-f168b7e565df admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:44.643316 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:44.687866 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 838/1672] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:30:44 2022] POST /identity/v3/users => generated 283 bytes in 541 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:44.693706 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf458a-70c7-4460-aca7-9257014d08c7 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:44.716861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf458a-70c7-4460-aca7-9257014d08c7 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:44.734025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba445a7-26ab-455b-8877-011ffa551b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:44.737475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba445a7-26ab-455b-8877-011ffa551b98 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:44.737475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba445a7-26ab-455b-8877-011ffa551b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:44.737885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba445a7-26ab-455b-8877-011ffa551b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:44.738575 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba445a7-26ab-455b-8877-011ffa551b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:44.747520 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba445a7-26ab-455b-8877-011ffa551b98 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:44.786235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 835/1673] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:30:44.792696 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45539a-512e-40c6-9fae-f168b7e565df admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:44.813120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45539a-512e-40c6-9fae-f168b7e565df admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:44.832450 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31930c-d053-4913-b089-27539a9a690e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:44.834625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31930c-d053-4913-b089-27539a9a690e admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:44.836367 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31930c-d053-4913-b089-27539a9a690e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:44.836367 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31930c-d053-4913-b089-27539a9a690e admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:44.838123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31930c-d053-4913-b089-27539a9a690e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:44.841772 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31930c-d053-4913-b089-27539a9a690e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:44.887524 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba445a7-26ab-455b-8877-011ffa551b98 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:44.895959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 839/1674] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:44.946570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba445a7-26ab-455b-8877-011ffa551b98 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:44.974346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622826f-8167-4e89-acb6-accebd174d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:44.984768 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622826f-8167-4e89-acb6-accebd174d8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:44.991180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622826f-8167-4e89-acb6-accebd174d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:44.991568 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622826f-8167-4e89-acb6-accebd174d8c admin admin] PATH_INFO: `/v3/projects/4f5c8a2e9aa64ccebb84e0a557d15e23/users/8c98e6ca34c8431d9740f03968f377b7/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:45.053631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4622826f-8167-4e89-acb6-accebd174d8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f5c8a2e9aa64ccebb84e0a557d15e23, user_id=8c98e6ca34c8431d9740f03968f377b7, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:45.058111 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4622826f-8167-4e89-acb6-accebd174d8c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:45.093347 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 836/1675] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:44 2022] PUT /identity/v3/projects/4f5c8a2e9aa64ccebb84e0a557d15e23/users/8c98e6ca34c8431d9740f03968f377b7/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:45.121004 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a7f4b-936e-4a1a-9656-da089b2ecdc1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:45.121613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a7f4b-936e-4a1a-9656-da089b2ecdc1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:45.122220 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a7f4b-936e-4a1a-9656-da089b2ecdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:45.204604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-953a7f4b-936e-4a1a-9656-da089b2ecdc1 None None] MFA Rules not processed for user `8c98e6ca34c8431d9740f03968f377b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:45.207532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-953a7f4b-936e-4a1a-9656-da089b2ecdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:45.471647 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 840/1676] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:45.476642 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622826f-8167-4e89-acb6-accebd174d8c admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:45.538073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:45.607973 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a7f4b-936e-4a1a-9656-da089b2ecdc1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:45.609745 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-953a7f4b-936e-4a1a-9656-da089b2ecdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:45.829073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a7f4b-936e-4a1a-9656-da089b2ecdc1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:45.846508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63bab9-7128-4e47-91da-e06c42971c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:45.847628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63bab9-7128-4e47-91da-e06c42971c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:45.848203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63bab9-7128-4e47-91da-e06c42971c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:45.848606 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63bab9-7128-4e47-91da-e06c42971c2d admin admin] PATH_INFO: `/v3/projects/dd79d3d9a6a846118b918fae7a1384b3` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:45.858881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63bab9-7128-4e47-91da-e06c42971c2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd79d3d9a6a846118b918fae7a1384b3)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:45.861168 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63bab9-7128-4e47-91da-e06c42971c2d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:45.971649 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fc63bab9-7128-4e47-91da-e06c42971c2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd79d3d9a6a846118b918fae7a1384b3', 'request_id': 'req-fc63bab9-7128-4e47-91da-e06c42971c2d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:45.980968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622826f-8167-4e89-acb6-accebd174d8c admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:45.997557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fc63bab9-7128-4e47-91da-e06c42971c2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd79d3d9a6a846118b918fae7a1384b3', 'request_id': 'req-fc63bab9-7128-4e47-91da-e06c42971c2d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:46.009133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fc63bab9-7128-4e47-91da-e06c42971c2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd79d3d9a6a846118b918fae7a1384b3', 'request_id': 'req-fc63bab9-7128-4e47-91da-e06c42971c2d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:46.022859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:46.032674 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fc63bab9-7128-4e47-91da-e06c42971c2d admin admin] The token cache is being invalidate because project dd79d3d9a6a846118b918fae7a1384b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:30:46.033750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fc63bab9-7128-4e47-91da-e06c42971c2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:46.034695 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fc63bab9-7128-4e47-91da-e06c42971c2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:46.036945 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 841/1677] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:30:45 2022] DELETE /identity/v3/projects/dd79d3d9a6a846118b918fae7a1384b3 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:30:46.048064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63bab9-7128-4e47-91da-e06c42971c2d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:46.053730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fc63bab9-7128-4e47-91da-e06c42971c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:46.162544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63bab9-7128-4e47-91da-e06c42971c2d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:46.171382 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbe384-bc13-4c6c-bb2d-6b84328f4622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:46.171696 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e923f-afd3-47d7-955b-9d0d56ce4150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:46.172145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e923f-afd3-47d7-955b-9d0d56ce4150 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:46.172437 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e923f-afd3-47d7-955b-9d0d56ce4150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:46.172728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e923f-afd3-47d7-955b-9d0d56ce4150 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:46.172930 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbe384-bc13-4c6c-bb2d-6b84328f4622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:46.173428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbe384-bc13-4c6c-bb2d-6b84328f4622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:46.173872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbe384-bc13-4c6c-bb2d-6b84328f4622 admin admin] PATH_INFO: `/v3/users/f338f2df615d489c91c23e5e291693d2` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:46.174990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e923f-afd3-47d7-955b-9d0d56ce4150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:46.175713 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dbe384-bc13-4c6c-bb2d-6b84328f4622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f338f2df615d489c91c23e5e291693d2)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:46.178426 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dbe384-bc13-4c6c-bb2d-6b84328f4622 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:46.191328 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e923f-afd3-47d7-955b-9d0d56ce4150 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:46.278951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 837/1678] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:30:45 2022] POST /identity/v3/projects => generated 363 bytes in 803 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:46.280739 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e923f-afd3-47d7-955b-9d0d56ce4150 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:46.286484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-684e923f-afd3-47d7-955b-9d0d56ce4150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:46.370301 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-90dbe384-bc13-4c6c-bb2d-6b84328f4622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f338f2df615d489c91c23e5e291693d2'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:46.399465 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-90dbe384-bc13-4c6c-bb2d-6b84328f4622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f338f2df615d489c91c23e5e291693d2'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:46.419698 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-90dbe384-bc13-4c6c-bb2d-6b84328f4622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f338f2df615d489c91c23e5e291693d2'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:46.420966 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-90dbe384-bc13-4c6c-bb2d-6b84328f4622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f338f2df615d489c91c23e5e291693d2'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:46.423009 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-90dbe384-bc13-4c6c-bb2d-6b84328f4622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f338f2df615d489c91c23e5e291693d2'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:46.427301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e923f-afd3-47d7-955b-9d0d56ce4150 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:46.451795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d105d1-0d89-457f-830c-116c5a8a66eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:46.453038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d105d1-0d89-457f-830c-116c5a8a66eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:46.453570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d105d1-0d89-457f-830c-116c5a8a66eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:46.453981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d105d1-0d89-457f-830c-116c5a8a66eb admin admin] PATH_INFO: `/v3/projects/95990ba77ff749e0aa91bed9d1608e43` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:46.454802 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 842/1679] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:30:46 2022] DELETE /identity/v3/users/f338f2df615d489c91c23e5e291693d2 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:46.456871 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d105d1-0d89-457f-830c-116c5a8a66eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=95990ba77ff749e0aa91bed9d1608e43)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:46.457341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbe384-bc13-4c6c-bb2d-6b84328f4622 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:46.459540 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d105d1-0d89-457f-830c-116c5a8a66eb admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:46.462640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-90dbe384-bc13-4c6c-bb2d-6b84328f4622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:46.595825 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-25d105d1-0d89-457f-830c-116c5a8a66eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95990ba77ff749e0aa91bed9d1608e43', 'request_id': 'req-25d105d1-0d89-457f-830c-116c5a8a66eb'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:46.604135 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:46.609001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-25d105d1-0d89-457f-830c-116c5a8a66eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95990ba77ff749e0aa91bed9d1608e43', 'request_id': 'req-25d105d1-0d89-457f-830c-116c5a8a66eb'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:46.625200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-25d105d1-0d89-457f-830c-116c5a8a66eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95990ba77ff749e0aa91bed9d1608e43', 'request_id': 'req-25d105d1-0d89-457f-830c-116c5a8a66eb'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:46.646843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-25d105d1-0d89-457f-830c-116c5a8a66eb admin admin] The token cache is being invalidate because project 95990ba77ff749e0aa91bed9d1608e43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:30:46.647196 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-25d105d1-0d89-457f-830c-116c5a8a66eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:46.649132 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-25d105d1-0d89-457f-830c-116c5a8a66eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:46.656600 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 838/1680] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:30:46 2022] DELETE /identity/v3/projects/95990ba77ff749e0aa91bed9d1608e43 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:46.658505 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f3010-79cf-42b7-8fcd-f0024f1d9feb None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:46.658505 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f3010-79cf-42b7-8fcd-f0024f1d9feb None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:46.658828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f3010-79cf-42b7-8fcd-f0024f1d9feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:46.712064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-953f3010-79cf-42b7-8fcd-f0024f1d9feb None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:46.713680 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-953f3010-79cf-42b7-8fcd-f0024f1d9feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:46.774206 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbe384-bc13-4c6c-bb2d-6b84328f4622 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:46.839848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 839/1681] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:46.844215 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b668d-afab-49ae-895a-318f5d2c3ac8 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:46.844518 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b668d-afab-49ae-895a-318f5d2c3ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:46.844799 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b668d-afab-49ae-895a-318f5d2c3ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:46.874738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-249b668d-afab-49ae-895a-318f5d2c3ac8 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:46.876967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-249b668d-afab-49ae-895a-318f5d2c3ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:46.906058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 840/1682] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:46.907096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b668d-afab-49ae-895a-318f5d2c3ac8 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:46.928300 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b668d-afab-49ae-895a-318f5d2c3ac8 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:46.934809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:46.947423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94bb898-4d64-40ee-b622-6e0dec1d5275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:46.949756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bb898-4d64-40ee-b622-6e0dec1d5275 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:46.950070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bb898-4d64-40ee-b622-6e0dec1d5275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:46.950442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bb898-4d64-40ee-b622-6e0dec1d5275 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:46.951722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94bb898-4d64-40ee-b622-6e0dec1d5275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:30:46.952577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94bb898-4d64-40ee-b622-6e0dec1d5275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:46.955445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94bb898-4d64-40ee-b622-6e0dec1d5275 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:46.997582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 841/1683] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:30:47.021731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94bb898-4d64-40ee-b622-6e0dec1d5275 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:47.060368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94bb898-4d64-40ee-b622-6e0dec1d5275 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:47.074248 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29930b0f-8344-487e-bf60-b9e8f3ee92ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:47.079287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29930b0f-8344-487e-bf60-b9e8f3ee92ba admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:47.081525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29930b0f-8344-487e-bf60-b9e8f3ee92ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:47.081525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29930b0f-8344-487e-bf60-b9e8f3ee92ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:47.082974 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29930b0f-8344-487e-bf60-b9e8f3ee92ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:47.085323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29930b0f-8344-487e-bf60-b9e8f3ee92ba admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:47.171790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 842/1684] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:30:47 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:47.171790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29930b0f-8344-487e-bf60-b9e8f3ee92ba admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:47.174648 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-29930b0f-8344-487e-bf60-b9e8f3ee92ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:47.215684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2adac-fa34-43c9-985c-af89e2e2f92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:47.216827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2adac-fa34-43c9-985c-af89e2e2f92d admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:47.217334 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2adac-fa34-43c9-985c-af89e2e2f92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:47.217828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2adac-fa34-43c9-985c-af89e2e2f92d admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:47.219185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec2adac-fa34-43c9-985c-af89e2e2f92d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:47.232574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec2adac-fa34-43c9-985c-af89e2e2f92d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:47.353786 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 843/1685] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:30:46 2022] POST /identity/v3/users => generated 288 bytes in 893 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:47.353786 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2adac-fa34-43c9-985c-af89e2e2f92d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:47.379796 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2adac-fa34-43c9-985c-af89e2e2f92d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:47.430940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29930b0f-8344-487e-bf60-b9e8f3ee92ba admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:47.455167 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db7541-1577-44f3-86f9-a0ab4591e059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:47.457154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db7541-1577-44f3-86f9-a0ab4591e059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:47.457696 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db7541-1577-44f3-86f9-a0ab4591e059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:47.458197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db7541-1577-44f3-86f9-a0ab4591e059 admin admin] PATH_INFO: `/v3/projects/4a20802a2cc846719c516a2603daeb04` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:47.462026 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db7541-1577-44f3-86f9-a0ab4591e059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a20802a2cc846719c516a2603daeb04)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:47.475867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db7541-1577-44f3-86f9-a0ab4591e059 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:47.656039 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ab018-4036-40b4-b554-146a812c9857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:47.659633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ab018-4036-40b4-b554-146a812c9857 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:47.660203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ab018-4036-40b4-b554-146a812c9857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:47.665780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ab018-4036-40b4-b554-146a812c9857 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:47.667220 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ab018-4036-40b4-b554-146a812c9857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:47.685405 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ab018-4036-40b4-b554-146a812c9857 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:47.726798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:47.742689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-42db7541-1577-44f3-86f9-a0ab4591e059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a20802a2cc846719c516a2603daeb04', 'request_id': 'req-42db7541-1577-44f3-86f9-a0ab4591e059'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:47.774360 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-42db7541-1577-44f3-86f9-a0ab4591e059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a20802a2cc846719c516a2603daeb04', 'request_id': 'req-42db7541-1577-44f3-86f9-a0ab4591e059'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:47.798662 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-42db7541-1577-44f3-86f9-a0ab4591e059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a20802a2cc846719c516a2603daeb04', 'request_id': 'req-42db7541-1577-44f3-86f9-a0ab4591e059'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:47.816996 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 844/1686] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:30:47 2022] POST /identity/v3/users => generated 291 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:47.825877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ab018-4036-40b4-b554-146a812c9857 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:47.851392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fc3ab018-4036-40b4-b554-146a812c9857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:47.860517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-42db7541-1577-44f3-86f9-a0ab4591e059 admin admin] The token cache is being invalidate because project 4a20802a2cc846719c516a2603daeb04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:30:47.860517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-42db7541-1577-44f3-86f9-a0ab4591e059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:47.863119 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-42db7541-1577-44f3-86f9-a0ab4591e059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:47.873712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 843/1687] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:30:47 2022] DELETE /identity/v3/projects/4a20802a2cc846719c516a2603daeb04 => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:47.878954 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db7541-1577-44f3-86f9-a0ab4591e059 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:47.893259 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-42db7541-1577-44f3-86f9-a0ab4591e059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:47.945098 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ab018-4036-40b4-b554-146a812c9857 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:48.124902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8491e-cfb1-4953-a3f0-6438196d2c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:48.127285 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8491e-cfb1-4953-a3f0-6438196d2c30 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:48.127629 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8491e-cfb1-4953-a3f0-6438196d2c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:48.127942 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8491e-cfb1-4953-a3f0-6438196d2c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:48.128662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8491e-cfb1-4953-a3f0-6438196d2c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:48.130605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8491e-cfb1-4953-a3f0-6438196d2c30 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:48.175695 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 845/1688] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:48.176446 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9af735-d64a-4d30-858b-127968b38527 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:48.176826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9af735-d64a-4d30-858b-127968b38527 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:48.177110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9af735-d64a-4d30-858b-127968b38527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:48.237857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4b9af735-d64a-4d30-858b-127968b38527 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:48.241900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4b9af735-d64a-4d30-858b-127968b38527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:48.293238 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 846/1689] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:48.294401 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9af735-d64a-4d30-858b-127968b38527 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:48.301892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4b9af735-d64a-4d30-858b-127968b38527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:48.339233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db7541-1577-44f3-86f9-a0ab4591e059 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:48.360114 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9af735-d64a-4d30-858b-127968b38527 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:48.362813 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65050932-6b06-44c5-b067-212eee73afe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:48.364649 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65050932-6b06-44c5-b067-212eee73afe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:48.365491 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65050932-6b06-44c5-b067-212eee73afe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:48.366184 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65050932-6b06-44c5-b067-212eee73afe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:48.371715 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65050932-6b06-44c5-b067-212eee73afe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:48.378371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65050932-6b06-44c5-b067-212eee73afe8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:48.389058 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27edcf-9648-4351-879b-2599c002972c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:48.394927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27edcf-9648-4351-879b-2599c002972c admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:48.395279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27edcf-9648-4351-879b-2599c002972c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:48.395572 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27edcf-9648-4351-879b-2599c002972c admin admin] PATH_INFO: `/v3/projects/d8c43c7566ba444dbc0e63a9f6c7df71/users/0396b085da804b6099905d150d722bf6/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:48.438245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 844/1690] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 564 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:48.441380 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27edcf-9648-4351-879b-2599c002972c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8c43c7566ba444dbc0e63a9f6c7df71, user_id=0396b085da804b6099905d150d722bf6, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:48.446341 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387cf23-475f-43bc-b83c-dd180f445666 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:48.449397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387cf23-475f-43bc-b83c-dd180f445666 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:48.449630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27edcf-9648-4351-879b-2599c002972c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:48.451874 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387cf23-475f-43bc-b83c-dd180f445666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:48.522652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 847/1691] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:48 2022] PUT /identity/v3/projects/d8c43c7566ba444dbc0e63a9f6c7df71/users/0396b085da804b6099905d150d722bf6/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:48.524629 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27edcf-9648-4351-879b-2599c002972c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:48.546730 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-1387cf23-475f-43bc-b83c-dd180f445666 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:48.548778 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1387cf23-475f-43bc-b83c-dd180f445666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:48.587266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:48.821182 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27edcf-9648-4351-879b-2599c002972c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:48.843366 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c8915-dcc8-47c1-b153-a5b69ef1b28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:48.844916 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c8915-dcc8-47c1-b153-a5b69ef1b28e admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:48.845073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c8915-dcc8-47c1-b153-a5b69ef1b28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:48.845742 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c8915-dcc8-47c1-b153-a5b69ef1b28e admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:48.846914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c8915-dcc8-47c1-b153-a5b69ef1b28e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:48.864552 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c8915-dcc8-47c1-b153-a5b69ef1b28e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:48.923366 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 848/1692] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:48.923366 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c8915-dcc8-47c1-b153-a5b69ef1b28e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:48.952674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 845/1693] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 513 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:48.963856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81024053-a00d-4f93-9c12-9f5b9390df77 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:48.964356 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81024053-a00d-4f93-9c12-9f5b9390df77 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:48.973390 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81024053-a00d-4f93-9c12-9f5b9390df77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:48.981305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c8915-dcc8-47c1-b153-a5b69ef1b28e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:49.006987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa48a0-0e39-4f9c-82f8-fd9d7970c6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:49.009115 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa48a0-0e39-4f9c-82f8-fd9d7970c6fe admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:49.011240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa48a0-0e39-4f9c-82f8-fd9d7970c6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:49.011759 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa48a0-0e39-4f9c-82f8-fd9d7970c6fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:49.012751 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa48a0-0e39-4f9c-82f8-fd9d7970c6fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:49.016397 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa48a0-0e39-4f9c-82f8-fd9d7970c6fe admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:49.045520 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 849/1694] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:49.051377 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa48a0-0e39-4f9c-82f8-fd9d7970c6fe admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:49.083830 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa48a0-0e39-4f9c-82f8-fd9d7970c6fe admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:49.090453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-81024053-a00d-4f93-9c12-9f5b9390df77 None None] MFA Rules not processed for user `aee77a3ed91445ab8b7f58e7f8bc348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:49.101620 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd9b53-dbae-40f0-acae-1330f268ac9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:49.103172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd9b53-dbae-40f0-acae-1330f268ac9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:49.107069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd9b53-dbae-40f0-acae-1330f268ac9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:49.107069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd9b53-dbae-40f0-acae-1330f268ac9a admin admin] PATH_INFO: `/v3/projects/f8725989113f47fd839541e2c213ba9a/users/a1dd172493724e959fdb915c6dc6b214/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:49.113602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-81024053-a00d-4f93-9c12-9f5b9390df77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:49.139941 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd9b53-dbae-40f0-acae-1330f268ac9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8725989113f47fd839541e2c213ba9a, user_id=a1dd172493724e959fdb915c6dc6b214, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:49.143411 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd9b53-dbae-40f0-acae-1330f268ac9a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:49.184674 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 850/1695] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:49 2022] PUT /identity/v3/projects/f8725989113f47fd839541e2c213ba9a/users/a1dd172493724e959fdb915c6dc6b214/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:49.186130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd9b53-dbae-40f0-acae-1330f268ac9a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:49.369670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd9b53-dbae-40f0-acae-1330f268ac9a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:49.400238 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c7e52-13a6-4952-aab6-772e79297e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:49.404509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c7e52-13a6-4952-aab6-772e79297e36 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:49.404796 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c7e52-13a6-4952-aab6-772e79297e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:49.405080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c7e52-13a6-4952-aab6-772e79297e36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:49.422488 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c7e52-13a6-4952-aab6-772e79297e36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:30:49.423849 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c7e52-13a6-4952-aab6-772e79297e36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:49.426893 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c7e52-13a6-4952-aab6-772e79297e36 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:49.446421 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 846/1696] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 06:30:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:49.448132 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81024053-a00d-4f93-9c12-9f5b9390df77 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:49.490159 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 851/1697] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:30:49.493563 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a122fc-24b7-40dd-806b-6fafd27b738d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:49.494827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a122fc-24b7-40dd-806b-6fafd27b738d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:49.495635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a122fc-24b7-40dd-806b-6fafd27b738d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:49.535341 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81024053-a00d-4f93-9c12-9f5b9390df77 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:49.605419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb55b78-779e-4938-a404-4b898e72b27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:49.605419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb55b78-779e-4938-a404-4b898e72b27c admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:49.605419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb55b78-779e-4938-a404-4b898e72b27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:49.605419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb55b78-779e-4938-a404-4b898e72b27c admin admin] PATH_INFO: `/v3/projects/d8c43c7566ba444dbc0e63a9f6c7df71/users/0396b085da804b6099905d150d722bf6/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:49.606592 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-34a122fc-24b7-40dd-806b-6fafd27b738d None None] MFA Rules not processed for user `a1dd172493724e959fdb915c6dc6b214`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:49.613549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-34a122fc-24b7-40dd-806b-6fafd27b738d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:49.623621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb55b78-779e-4938-a404-4b898e72b27c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8c43c7566ba444dbc0e63a9f6c7df71, user_id=0396b085da804b6099905d150d722bf6, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:49.651958 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb55b78-779e-4938-a404-4b898e72b27c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:49.715768 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 847/1698] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:49 2022] PUT /identity/v3/projects/d8c43c7566ba444dbc0e63a9f6c7df71/users/0396b085da804b6099905d150d722bf6/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:49.731370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb55b78-779e-4938-a404-4b898e72b27c admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:49.750188 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:49.974234 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:50.027045 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 852/1699] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 536 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:50.032856 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad929cd-cfc6-497d-a549-f2038ae6689f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:50.037105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad929cd-cfc6-497d-a549-f2038ae6689f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:50.037634 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad929cd-cfc6-497d-a549-f2038ae6689f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:50.152339 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9ad929cd-cfc6-497d-a549-f2038ae6689f None None] MFA Rules not processed for user `0396b085da804b6099905d150d722bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:50.154659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9ad929cd-cfc6-497d-a549-f2038ae6689f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:50.374823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb55b78-779e-4938-a404-4b898e72b27c admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:50.400064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882872e8-6745-4658-916d-8c8c6fc23399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:50.404205 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882872e8-6745-4658-916d-8c8c6fc23399 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:50.407459 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882872e8-6745-4658-916d-8c8c6fc23399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:50.408855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882872e8-6745-4658-916d-8c8c6fc23399 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:50.412797 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882872e8-6745-4658-916d-8c8c6fc23399 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:50.421360 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882872e8-6745-4658-916d-8c8c6fc23399 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:50.549211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 853/1700] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:50 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 519 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:50.559712 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad929cd-cfc6-497d-a549-f2038ae6689f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:50.564370 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9ad929cd-cfc6-497d-a549-f2038ae6689f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:50.588911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 848/1701] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:30:49 2022] POST /identity/v3/projects => generated 377 bytes in 864 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:50.590771 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882872e8-6745-4658-916d-8c8c6fc23399 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:50.859579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad929cd-cfc6-497d-a549-f2038ae6689f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:50.864287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882872e8-6745-4658-916d-8c8c6fc23399 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:50.876838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70673f-59a6-48aa-9663-bfd11ffcb640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:50.877604 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70673f-59a6-48aa-9663-bfd11ffcb640 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:50.877878 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70673f-59a6-48aa-9663-bfd11ffcb640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:50.878185 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70673f-59a6-48aa-9663-bfd11ffcb640 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:50.881917 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70673f-59a6-48aa-9663-bfd11ffcb640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:50.884353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70673f-59a6-48aa-9663-bfd11ffcb640 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:50.981430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 849/1702] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:30:50 2022] POST /identity/v3/projects => generated 381 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:50.986033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70673f-59a6-48aa-9663-bfd11ffcb640 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:51.058390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6dc61e-c552-44bd-aa2d-c5f90b81a780 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc205d053d54506bb6c9227e4a1b044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:51.064295 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dc61e-c552-44bd-aa2d-c5f90b81a780 service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:51.064684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dc61e-c552-44bd-aa2d-c5f90b81a780 service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:51.064979 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dc61e-c552-44bd-aa2d-c5f90b81a780 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:51.165611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:51.310412 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70673f-59a6-48aa-9663-bfd11ffcb640 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:51.349921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d5bda-0736-4fa5-846c-adf804666544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:51.354814 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d5bda-0736-4fa5-846c-adf804666544 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:51.355523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d5bda-0736-4fa5-846c-adf804666544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:51.356183 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d5bda-0736-4fa5-846c-adf804666544 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:51.357774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d5bda-0736-4fa5-846c-adf804666544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:51.372832 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d5bda-0736-4fa5-846c-adf804666544 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:51.384888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6dc61e-c552-44bd-aa2d-c5f90b81a780 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:51.402471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6dc61e-c552-44bd-aa2d-c5f90b81a780 service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:51.449137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 854/1703] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 06:30:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 899 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 06:30:51.450231 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6dc61e-c552-44bd-aa2d-c5f90b81a780 service swift] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:51.458957 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4c6dc61e-c552-44bd-aa2d-c5f90b81a780 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:51.486090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 850/1704] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:30:50 2022] POST /identity/v3/users => generated 289 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:51.486090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d5bda-0736-4fa5-846c-adf804666544 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:51.530631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:51.563352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d5bda-0736-4fa5-846c-adf804666544 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:51.591837 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ebd8a-b944-46ed-a8ac-9ee94bcd2f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:51.592383 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ebd8a-b944-46ed-a8ac-9ee94bcd2f28 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:51.592570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ebd8a-b944-46ed-a8ac-9ee94bcd2f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:51.596525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ebd8a-b944-46ed-a8ac-9ee94bcd2f28 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:51.597247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ebd8a-b944-46ed-a8ac-9ee94bcd2f28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:51.609693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ebd8a-b944-46ed-a8ac-9ee94bcd2f28 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:51.635486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6dc61e-c552-44bd-aa2d-c5f90b81a780 service swift] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:51.672168 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 851/1705] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:30:51 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:51.673320 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ebd8a-b944-46ed-a8ac-9ee94bcd2f28 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:51.722260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ebd8a-b944-46ed-a8ac-9ee94bcd2f28 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:51.747191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600d93b-51d8-4473-bfc2-2dd851820155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:51.747909 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600d93b-51d8-4473-bfc2-2dd851820155 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:51.748204 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600d93b-51d8-4473-bfc2-2dd851820155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:51.748615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600d93b-51d8-4473-bfc2-2dd851820155 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:51.749232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600d93b-51d8-4473-bfc2-2dd851820155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:51.750984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600d93b-51d8-4473-bfc2-2dd851820155 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:51.806329 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 852/1706] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:51.810696 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600d93b-51d8-4473-bfc2-2dd851820155 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:51.817966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b600d93b-51d8-4473-bfc2-2dd851820155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:51.857166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600d93b-51d8-4473-bfc2-2dd851820155 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:51.872058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65877023-3f41-46e6-a0cf-9e741a0b268f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:51.873089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65877023-3f41-46e6-a0cf-9e741a0b268f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:51.875674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65877023-3f41-46e6-a0cf-9e741a0b268f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:51.875956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65877023-3f41-46e6-a0cf-9e741a0b268f admin admin] PATH_INFO: `/v3/projects/6fadf42a41e7451b9e3d9845f8c5cfa7` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:51.888001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65877023-3f41-46e6-a0cf-9e741a0b268f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fadf42a41e7451b9e3d9845f8c5cfa7)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:51.894359 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65877023-3f41-46e6-a0cf-9e741a0b268f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:51.901174 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da3007-2d1c-4b4c-be4d-731c1044fdb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:51.913326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da3007-2d1c-4b4c-be4d-731c1044fdb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:51.913804 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da3007-2d1c-4b4c-be4d-731c1044fdb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:51.914236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da3007-2d1c-4b4c-be4d-731c1044fdb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:52.032248 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-65877023-3f41-46e6-a0cf-9e741a0b268f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fadf42a41e7451b9e3d9845f8c5cfa7', 'request_id': 'req-65877023-3f41-46e6-a0cf-9e741a0b268f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:52.051516 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-65877023-3f41-46e6-a0cf-9e741a0b268f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fadf42a41e7451b9e3d9845f8c5cfa7', 'request_id': 'req-65877023-3f41-46e6-a0cf-9e741a0b268f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:52.067247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-65877023-3f41-46e6-a0cf-9e741a0b268f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fadf42a41e7451b9e3d9845f8c5cfa7', 'request_id': 'req-65877023-3f41-46e6-a0cf-9e741a0b268f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:52.095025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-65877023-3f41-46e6-a0cf-9e741a0b268f admin admin] The token cache is being invalidate because project 6fadf42a41e7451b9e3d9845f8c5cfa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:30:52.095408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-65877023-3f41-46e6-a0cf-9e741a0b268f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:52.097866 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-65877023-3f41-46e6-a0cf-9e741a0b268f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:52.101384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 853/1707] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:30:51 2022] DELETE /identity/v3/projects/6fadf42a41e7451b9e3d9845f8c5cfa7 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:52.102412 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65877023-3f41-46e6-a0cf-9e741a0b268f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:52.106950 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-65877023-3f41-46e6-a0cf-9e741a0b268f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:52.251474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da3007-2d1c-4b4c-be4d-731c1044fdb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:52.253325 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da3007-2d1c-4b4c-be4d-731c1044fdb9 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:52.294028 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 855/1708] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:51 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 844 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:52.295992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da3007-2d1c-4b4c-be4d-731c1044fdb9 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:52.333409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da3007-2d1c-4b4c-be4d-731c1044fdb9 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:52.408254 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65877023-3f41-46e6-a0cf-9e741a0b268f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:52.444565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33671a71-a758-4654-b14c-12da4dd1bdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:52.446115 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33671a71-a758-4654-b14c-12da4dd1bdd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:52.446501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33671a71-a758-4654-b14c-12da4dd1bdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:52.446824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33671a71-a758-4654-b14c-12da4dd1bdd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:52.447513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33671a71-a758-4654-b14c-12da4dd1bdd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:52.448407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:52.449208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33671a71-a758-4654-b14c-12da4dd1bdd9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:52.550373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 854/1709] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:52.560163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfbe18-f82d-4ca2-bede-666d47e77fb9 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:52.561932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfbe18-f82d-4ca2-bede-666d47e77fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:52.564781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfbe18-f82d-4ca2-bede-666d47e77fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:52.576311 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d76d7f-1ea8-492c-bd83-419fcfaa72e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:52.581171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d76d7f-1ea8-492c-bd83-419fcfaa72e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:52.586875 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d76d7f-1ea8-492c-bd83-419fcfaa72e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:52.586875 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d76d7f-1ea8-492c-bd83-419fcfaa72e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:52.587142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d76d7f-1ea8-492c-bd83-419fcfaa72e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:52.588822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d76d7f-1ea8-492c-bd83-419fcfaa72e4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:52.649475 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 856/1710] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:52.651494 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d76d7f-1ea8-492c-bd83-419fcfaa72e4 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:52.688282 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d76d7f-1ea8-492c-bd83-419fcfaa72e4 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:52.697175 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-05cfbe18-f82d-4ca2-bede-666d47e77fb9 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:52.706492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-05cfbe18-f82d-4ca2-bede-666d47e77fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:52.726557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0ca1d-a489-4019-88a9-6b51f8970d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:52.733421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0ca1d-a489-4019-88a9-6b51f8970d76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:52.738828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0ca1d-a489-4019-88a9-6b51f8970d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:52.740915 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0ca1d-a489-4019-88a9-6b51f8970d76 admin admin] PATH_INFO: `/v3/projects/1925eeac247841b98efec535b1cdbe4f/users/45c1210cf3154927ad8f3d307093b684/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:52.775732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 855/1711] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:52.778152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cfbe18-f82d-4ca2-bede-666d47e77fb9 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:52.796905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-05cfbe18-f82d-4ca2-bede-666d47e77fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:52.830718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0ca1d-a489-4019-88a9-6b51f8970d76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1925eeac247841b98efec535b1cdbe4f, user_id=45c1210cf3154927ad8f3d307093b684, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:52.850643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0ca1d-a489-4019-88a9-6b51f8970d76 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:52.881729 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cfbe18-f82d-4ca2-bede-666d47e77fb9 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:52.910738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043440d5-fd81-42bb-b103-82cbd7a63f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:52.922804 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043440d5-fd81-42bb-b103-82cbd7a63f06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:52.925996 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043440d5-fd81-42bb-b103-82cbd7a63f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:52.926536 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043440d5-fd81-42bb-b103-82cbd7a63f06 admin admin] PATH_INFO: `/v3/projects/ebbae5a0810e468ebe0bc3234a635f4a/users/a185f5eca38344a99a5333cbac424015/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:52.934524 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 857/1712] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:52 2022] PUT /identity/v3/projects/1925eeac247841b98efec535b1cdbe4f/users/45c1210cf3154927ad8f3d307093b684/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:52.935635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf3ff1-2583-464b-a4f3-55393a1e7a46 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:52.935935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf3ff1-2583-464b-a4f3-55393a1e7a46 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:52.936224 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf3ff1-2583-464b-a4f3-55393a1e7a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:53.061244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043440d5-fd81-42bb-b103-82cbd7a63f06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebbae5a0810e468ebe0bc3234a635f4a, user_id=a185f5eca38344a99a5333cbac424015, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:53.082840 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043440d5-fd81-42bb-b103-82cbd7a63f06 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:53.119987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9bbf3ff1-2583-464b-a4f3-55393a1e7a46 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:53.121580 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9bbf3ff1-2583-464b-a4f3-55393a1e7a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:53.157910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 856/1713] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:52 2022] PUT /identity/v3/projects/ebbae5a0810e468ebe0bc3234a635f4a/users/a185f5eca38344a99a5333cbac424015/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:53.162588 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043440d5-fd81-42bb-b103-82cbd7a63f06 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:53.373104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:53.653614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 858/1714] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 719 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:53.664259 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df6eaf-597c-49ed-a846-e1ee4792b55b None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:53.664843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df6eaf-597c-49ed-a846-e1ee4792b55b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:53.672214 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df6eaf-597c-49ed-a846-e1ee4792b55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:53.674864 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043440d5-fd81-42bb-b103-82cbd7a63f06 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:53.705837 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499cdcd-a1fc-4dd4-865f-a0ecc8ecb109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:53.707882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499cdcd-a1fc-4dd4-865f-a0ecc8ecb109 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:53.710029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499cdcd-a1fc-4dd4-865f-a0ecc8ecb109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:53.713353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499cdcd-a1fc-4dd4-865f-a0ecc8ecb109 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:53.717584 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499cdcd-a1fc-4dd4-865f-a0ecc8ecb109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:53.724295 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499cdcd-a1fc-4dd4-865f-a0ecc8ecb109 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:53.789201 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 857/1715] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 630 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:53.790299 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499cdcd-a1fc-4dd4-865f-a0ecc8ecb109 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:53.823329 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499cdcd-a1fc-4dd4-865f-a0ecc8ecb109 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:53.833565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f1df6eaf-597c-49ed-a846-e1ee4792b55b None None] MFA Rules not processed for user `a185f5eca38344a99a5333cbac424015`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:53.836753 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f1df6eaf-597c-49ed-a846-e1ee4792b55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:53.847769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022cdd3-5a63-43f9-a4fc-9bf3846cda13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:53.848488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022cdd3-5a63-43f9-a4fc-9bf3846cda13 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:53.848785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022cdd3-5a63-43f9-a4fc-9bf3846cda13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:53.849068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022cdd3-5a63-43f9-a4fc-9bf3846cda13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:53.849982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022cdd3-5a63-43f9-a4fc-9bf3846cda13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:30:53.850309 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022cdd3-5a63-43f9-a4fc-9bf3846cda13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:53.852235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022cdd3-5a63-43f9-a4fc-9bf3846cda13 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:53.996966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 858/1716] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:30:53.999641 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022cdd3-5a63-43f9-a4fc-9bf3846cda13 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:54.088181 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022cdd3-5a63-43f9-a4fc-9bf3846cda13 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:54.110586 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4a05b-f4d5-49f5-9918-4e17306c2bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:54.114266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4a05b-f4d5-49f5-9918-4e17306c2bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:54.115110 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4a05b-f4d5-49f5-9918-4e17306c2bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:54.118833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4a05b-f4d5-49f5-9918-4e17306c2bc0 admin admin] PATH_INFO: `/v3/projects/1925eeac247841b98efec535b1cdbe4f/users/45c1210cf3154927ad8f3d307093b684/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:54.123184 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4a05b-f4d5-49f5-9918-4e17306c2bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1925eeac247841b98efec535b1cdbe4f, user_id=45c1210cf3154927ad8f3d307093b684, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:54.176380 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4a05b-f4d5-49f5-9918-4e17306c2bc0 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:54.309334 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 859/1717] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:53 2022] PUT /identity/v3/projects/1925eeac247841b98efec535b1cdbe4f/users/45c1210cf3154927ad8f3d307093b684/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:54.315988 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4a05b-f4d5-49f5-9918-4e17306c2bc0 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:54.394901 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 859/1718] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 741 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:54.396339 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaa252-0cad-4063-b977-874b0aa2a8e0 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:54.396641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaa252-0cad-4063-b977-874b0aa2a8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:54.396921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaa252-0cad-4063-b977-874b0aa2a8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:54.601990 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d7eaa252-0cad-4063-b977-874b0aa2a8e0 None None] MFA Rules not processed for user `45c1210cf3154927ad8f3d307093b684`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:54.606179 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d7eaa252-0cad-4063-b977-874b0aa2a8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:54.847216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4a05b-f4d5-49f5-9918-4e17306c2bc0 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:54.850140 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:54.883044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca851a94-7cdd-426e-a4e7-2559f377fc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:54.883909 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca851a94-7cdd-426e-a4e7-2559f377fc49 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:54.888289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca851a94-7cdd-426e-a4e7-2559f377fc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:54.888289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca851a94-7cdd-426e-a4e7-2559f377fc49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:54.891739 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca851a94-7cdd-426e-a4e7-2559f377fc49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:54.902258 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca851a94-7cdd-426e-a4e7-2559f377fc49 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:55.005315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 860/1719] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:30:54 2022] POST /identity/v3/projects => generated 387 bytes in 695 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:55.011989 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca851a94-7cdd-426e-a4e7-2559f377fc49 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:55.015667 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ca851a94-7cdd-426e-a4e7-2559f377fc49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:55.080162 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 860/1720] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:54 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 685 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:55.086341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaa252-0cad-4063-b977-874b0aa2a8e0 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:55.090300 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d7eaa252-0cad-4063-b977-874b0aa2a8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:55.222778 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca851a94-7cdd-426e-a4e7-2559f377fc49 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:55.231935 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b1b76-4d23-4f8b-84ed-3b568dca8675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:55.233036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b1b76-4d23-4f8b-84ed-3b568dca8675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:55.235477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b1b76-4d23-4f8b-84ed-3b568dca8675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:55.235894 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b1b76-4d23-4f8b-84ed-3b568dca8675 admin admin] PATH_INFO: `/v3/users/8c98e6ca34c8431d9740f03968f377b7` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:55.244919 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b1b76-4d23-4f8b-84ed-3b568dca8675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c98e6ca34c8431d9740f03968f377b7)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:55.254171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b1b76-4d23-4f8b-84ed-3b568dca8675 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:55.272063 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaa252-0cad-4063-b977-874b0aa2a8e0 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:55.340151 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:55.421235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-612b1b76-4d23-4f8b-84ed-3b568dca8675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c98e6ca34c8431d9740f03968f377b7'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:55.422204 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-612b1b76-4d23-4f8b-84ed-3b568dca8675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c98e6ca34c8431d9740f03968f377b7'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:55.444843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-612b1b76-4d23-4f8b-84ed-3b568dca8675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c98e6ca34c8431d9740f03968f377b7'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:55.446050 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-612b1b76-4d23-4f8b-84ed-3b568dca8675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c98e6ca34c8431d9740f03968f377b7'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:55.478820 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-612b1b76-4d23-4f8b-84ed-3b568dca8675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c98e6ca34c8431d9740f03968f377b7'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:30:55.524562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 861/1721] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:30:55 2022] DELETE /identity/v3/users/8c98e6ca34c8431d9740f03968f377b7 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:55.524562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b1b76-4d23-4f8b-84ed-3b568dca8675 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:55.528221 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-612b1b76-4d23-4f8b-84ed-3b568dca8675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:55.537401 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433636fd-0413-472b-8ba7-a1b3ea53c5ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:55.538966 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433636fd-0413-472b-8ba7-a1b3ea53c5ad service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:55.539434 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433636fd-0413-472b-8ba7-a1b3ea53c5ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:55.539908 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433636fd-0413-472b-8ba7-a1b3ea53c5ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:55.558625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-433636fd-0413-472b-8ba7-a1b3ea53c5ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:55.961391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b1b76-4d23-4f8b-84ed-3b568dca8675 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:55.980340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7f92d-3b44-42a4-9725-13094863142b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:55.983181 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7f92d-3b44-42a4-9725-13094863142b admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:55.984069 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7f92d-3b44-42a4-9725-13094863142b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:55.984884 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7f92d-3b44-42a4-9725-13094863142b admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:55.987497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7f92d-3b44-42a4-9725-13094863142b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:55.995253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7f92d-3b44-42a4-9725-13094863142b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:56.002191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433636fd-0413-472b-8ba7-a1b3ea53c5ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:56.003767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433636fd-0413-472b-8ba7-a1b3ea53c5ad service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:56.044681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 861/1722] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 964 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:56.069829 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433636fd-0413-472b-8ba7-a1b3ea53c5ad service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:56.073902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-433636fd-0413-472b-8ba7-a1b3ea53c5ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:56.074170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 862/1723] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:30:55 2022] POST /identity/v3/users => generated 294 bytes in 554 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:56.077637 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1845219-5d14-4492-b296-619ca1713526 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:56.079008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1845219-5d14-4492-b296-619ca1713526 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:56.080121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1845219-5d14-4492-b296-619ca1713526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:56.131713 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433636fd-0413-472b-8ba7-a1b3ea53c5ad service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:56.174223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c1845219-5d14-4492-b296-619ca1713526 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:56.178849 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c1845219-5d14-4492-b296-619ca1713526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:56.252758 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:56.258168 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 863/1724] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:56.264970 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1845219-5d14-4492-b296-619ca1713526 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:56.354177 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1845219-5d14-4492-b296-619ca1713526 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:56.409320 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8e627-afcd-4e8b-8dcd-7bcdd9b5f827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:56.410542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8e627-afcd-4e8b-8dcd-7bcdd9b5f827 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:56.413697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8e627-afcd-4e8b-8dcd-7bcdd9b5f827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:56.416426 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8e627-afcd-4e8b-8dcd-7bcdd9b5f827 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:56.425990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8e627-afcd-4e8b-8dcd-7bcdd9b5f827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:56.440728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8e627-afcd-4e8b-8dcd-7bcdd9b5f827 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:56.451046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75363fef-37dc-4c51-b494-dffe261a3962 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:56.458347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75363fef-37dc-4c51-b494-dffe261a3962 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:56.458817 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75363fef-37dc-4c51-b494-dffe261a3962 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:56.461714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75363fef-37dc-4c51-b494-dffe261a3962 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:56.477187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-75363fef-37dc-4c51-b494-dffe261a3962 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:56.557936 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 864/1725] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:56.559552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8e627-afcd-4e8b-8dcd-7bcdd9b5f827 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:56.576762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-29f8e627-afcd-4e8b-8dcd-7bcdd9b5f827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:57.245898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75363fef-37dc-4c51-b494-dffe261a3962 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:57.262681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75363fef-37dc-4c51-b494-dffe261a3962 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:57.297384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8e627-afcd-4e8b-8dcd-7bcdd9b5f827 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:57.342979 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7263d6-559a-4a9e-a1b2-05ef405e5ab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:57.346589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7263d6-559a-4a9e-a1b2-05ef405e5ab7 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:57.347659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7263d6-559a-4a9e-a1b2-05ef405e5ab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:57.352315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7263d6-559a-4a9e-a1b2-05ef405e5ab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:57.379020 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 862/1726] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 1334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:57.381557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569ba4-e0f8-42ba-826e-5a666f841982 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:57.382576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569ba4-e0f8-42ba-826e-5a666f841982 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:57.383918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569ba4-e0f8-42ba-826e-5a666f841982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:57.442175 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7263d6-559a-4a9e-a1b2-05ef405e5ab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:57.467181 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7263d6-559a-4a9e-a1b2-05ef405e5ab7 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:57.577444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-fb569ba4-e0f8-42ba-826e-5a666f841982 None None] MFA Rules not processed for user `226f4700a516415cbbd272fdfad7af58`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:57.584253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 865/1727] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 1026 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:57.588431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7263d6-559a-4a9e-a1b2-05ef405e5ab7 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:57.599912 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fb569ba4-e0f8-42ba-826e-5a666f841982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:57.690215 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7263d6-559a-4a9e-a1b2-05ef405e5ab7 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:57.724777 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 863/1728] 10.222.0.59 () {58 vars in 1015 bytes} [Thu May 19 06:30:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:57.745369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e163939-ea87-459c-9d20-f5d6d925cf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:57.753966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e163939-ea87-459c-9d20-f5d6d925cf92 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:57.760259 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e163939-ea87-459c-9d20-f5d6d925cf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:57.764342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e163939-ea87-459c-9d20-f5d6d925cf92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:57.773977 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e163939-ea87-459c-9d20-f5d6d925cf92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:57.818541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e163939-ea87-459c-9d20-f5d6d925cf92 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:57.849421 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:57.889013 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 866/1729] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:30:57.895911 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb569ba4-e0f8-42ba-826e-5a666f841982 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:57.902203 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e163939-ea87-459c-9d20-f5d6d925cf92 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:58.010331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb569ba4-e0f8-42ba-826e-5a666f841982 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:58.014707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e163939-ea87-459c-9d20-f5d6d925cf92 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:58.077627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4390edc-e9ff-44e6-9d09-977a2d633620 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:58.078284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca537a-be74-4493-8a09-3ad7f6f4b730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:58.088465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4390edc-e9ff-44e6-9d09-977a2d633620 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:58.090864 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca537a-be74-4493-8a09-3ad7f6f4b730 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:58.092090 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca537a-be74-4493-8a09-3ad7f6f4b730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:58.094528 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4390edc-e9ff-44e6-9d09-977a2d633620 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:58.099616 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4390edc-e9ff-44e6-9d09-977a2d633620 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:58.103396 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca537a-be74-4493-8a09-3ad7f6f4b730 admin admin] PATH_INFO: `/v3/projects/88e493edc5d946c295aab5d435a58cc4/users/161885bb40814d0da588589d2c45e8c2/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:58.123581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f4390edc-e9ff-44e6-9d09-977a2d633620 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:58.259192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca537a-be74-4493-8a09-3ad7f6f4b730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e493edc5d946c295aab5d435a58cc4, user_id=161885bb40814d0da588589d2c45e8c2, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:58.274207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca537a-be74-4493-8a09-3ad7f6f4b730 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:58.359500 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 864/1730] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:57 2022] PUT /identity/v3/projects/88e493edc5d946c295aab5d435a58cc4/users/161885bb40814d0da588589d2c45e8c2/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:30:58.369008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca537a-be74-4493-8a09-3ad7f6f4b730 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:58.499811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4390edc-e9ff-44e6-9d09-977a2d633620 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:58.503760 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4390edc-e9ff-44e6-9d09-977a2d633620 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:58.555786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 867/1731] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 652 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:58.557586 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8a80e-f76e-4ced-af21-945109d61e51 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:58.562986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8a80e-f76e-4ced-af21-945109d61e51 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:58.564961 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8a80e-f76e-4ced-af21-945109d61e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:58.722285 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-74f8a80e-f76e-4ced-af21-945109d61e51 None None] MFA Rules not processed for user `161885bb40814d0da588589d2c45e8c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:30:58.727229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-74f8a80e-f76e-4ced-af21-945109d61e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:30:59.002539 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 868/1732] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:30:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 445 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:30:59.009056 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8a80e-f76e-4ced-af21-945109d61e51 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:59.023494 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca537a-be74-4493-8a09-3ad7f6f4b730 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:59.068949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67573c86-cc8e-401f-8979-202a3b73151d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:59.076395 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67573c86-cc8e-401f-8979-202a3b73151d service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:59.076766 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67573c86-cc8e-401f-8979-202a3b73151d service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:59.077214 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67573c86-cc8e-401f-8979-202a3b73151d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:59.141672 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67573c86-cc8e-401f-8979-202a3b73151d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:59.180095 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67573c86-cc8e-401f-8979-202a3b73151d service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:59.212520 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8a80e-f76e-4ced-af21-945109d61e51 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:59.227291 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ccf273-3226-4b0f-9942-fc2559fb8780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:59.228720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccf273-3226-4b0f-9942-fc2559fb8780 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:59.229229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccf273-3226-4b0f-9942-fc2559fb8780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:59.229727 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccf273-3226-4b0f-9942-fc2559fb8780 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:59.230962 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ccf273-3226-4b0f-9942-fc2559fb8780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:59.237094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ccf273-3226-4b0f-9942-fc2559fb8780 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:59.275080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 865/1733] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:30:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 909 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:30:59.370451 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 869/1734] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:30:59 2022] POST /identity/v3/projects => generated 387 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:59.375194 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67573c86-cc8e-401f-8979-202a3b73151d service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:59.462804 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67573c86-cc8e-401f-8979-202a3b73151d service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:59.696792 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:30:59.736423 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954d689-c6cc-4a0e-bc79-3e7862ce0df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:59.737433 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954d689-c6cc-4a0e-bc79-3e7862ce0df5 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:59.737910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954d689-c6cc-4a0e-bc79-3e7862ce0df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:59.738370 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954d689-c6cc-4a0e-bc79-3e7862ce0df5 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:59.739883 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954d689-c6cc-4a0e-bc79-3e7862ce0df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:59.747116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954d689-c6cc-4a0e-bc79-3e7862ce0df5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:59.813677 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ccf273-3226-4b0f-9942-fc2559fb8780 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:59.816060 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 866/1735] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:30:59 2022] POST /identity/v3/users => generated 300 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:30:59.896670 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ccf273-3226-4b0f-9942-fc2559fb8780 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:30:59.922453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02cda1-7650-412b-81a0-f68650fb57c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:30:59.926699 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02cda1-7650-412b-81a0-f68650fb57c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:30:59.927230 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02cda1-7650-412b-81a0-f68650fb57c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:30:59.927693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02cda1-7650-412b-81a0-f68650fb57c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:30:59.930167 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02cda1-7650-412b-81a0-f68650fb57c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:30:59.934211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02cda1-7650-412b-81a0-f68650fb57c2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:30:59.983249 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954d689-c6cc-4a0e-bc79-3e7862ce0df5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:30:59.998566 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 870/1736] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:00.004051 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954d689-c6cc-4a0e-bc79-3e7862ce0df5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:00.018120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7209cbf-bb51-4eb8-b123-a34a48a0dd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:00.020796 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7209cbf-bb51-4eb8-b123-a34a48a0dd29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:00.021515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7209cbf-bb51-4eb8-b123-a34a48a0dd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:00.023626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7209cbf-bb51-4eb8-b123-a34a48a0dd29 admin admin] PATH_INFO: `/v3/projects/478d6acaf74341739715f97ea06ba41c/users/2fcebcd0e08a4e24adbd3e8591d61712/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:00.086859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7209cbf-bb51-4eb8-b123-a34a48a0dd29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=478d6acaf74341739715f97ea06ba41c, user_id=2fcebcd0e08a4e24adbd3e8591d61712, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:00.093746 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7209cbf-bb51-4eb8-b123-a34a48a0dd29 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:00.153709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 867/1737] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:30:59 2022] PUT /identity/v3/projects/478d6acaf74341739715f97ea06ba41c/users/2fcebcd0e08a4e24adbd3e8591d61712/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:00.154115 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02cda1-7650-412b-81a0-f68650fb57c2 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:00.164373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:00.191953 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02cda1-7650-412b-81a0-f68650fb57c2 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:00.599726 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53158ba3-7e26-42cc-8bb0-311d25c12be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:00.603437 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53158ba3-7e26-42cc-8bb0-311d25c12be9 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:00.606024 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53158ba3-7e26-42cc-8bb0-311d25c12be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:00.606649 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53158ba3-7e26-42cc-8bb0-311d25c12be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:00.608035 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53158ba3-7e26-42cc-8bb0-311d25c12be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:00.623369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53158ba3-7e26-42cc-8bb0-311d25c12be9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:00.693902 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 871/1738] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 541 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:00.703558 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7209cbf-bb51-4eb8-b123-a34a48a0dd29 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:00.735858 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7209cbf-bb51-4eb8-b123-a34a48a0dd29 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:00.755301 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73df37-7f24-4756-8702-d03cda656561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:00.761386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73df37-7f24-4756-8702-d03cda656561 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:00.761985 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73df37-7f24-4756-8702-d03cda656561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:00.762461 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73df37-7f24-4756-8702-d03cda656561 admin admin] PATH_INFO: `/v3/projects/478d6acaf74341739715f97ea06ba41c/users/2fcebcd0e08a4e24adbd3e8591d61712/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:00.798834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73df37-7f24-4756-8702-d03cda656561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=478d6acaf74341739715f97ea06ba41c, user_id=2fcebcd0e08a4e24adbd3e8591d61712, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:00.817315 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73df37-7f24-4756-8702-d03cda656561 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:00.893339 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 868/1739] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:00 2022] PUT /identity/v3/projects/478d6acaf74341739715f97ea06ba41c/users/2fcebcd0e08a4e24adbd3e8591d61712/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:00.907358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c0373-44e7-4746-b430-894827e0e73b None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:00.914175 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c0373-44e7-4746-b430-894827e0e73b None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:00.914751 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c0373-44e7-4746-b430-894827e0e73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:01.030862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d88c0373-44e7-4746-b430-894827e0e73b None None] MFA Rules not processed for user `2fcebcd0e08a4e24adbd3e8591d61712`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:01.032328 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d88c0373-44e7-4746-b430-894827e0e73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:01.426842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 872/1740] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 523 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:01.446452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73df37-7f24-4756-8702-d03cda656561 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:01.797872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73df37-7f24-4756-8702-d03cda656561 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:01.829228 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095229c-7adb-4014-9491-f0f1eafb4c0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:01.830422 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095229c-7adb-4014-9491-f0f1eafb4c0b service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:01.839843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095229c-7adb-4014-9491-f0f1eafb4c0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:01.840508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095229c-7adb-4014-9491-f0f1eafb4c0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:01.948961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095229c-7adb-4014-9491-f0f1eafb4c0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:01.997542 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095229c-7adb-4014-9491-f0f1eafb4c0b service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:02.051005 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 869/1741] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:02.677090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c0373-44e7-4746-b430-894827e0e73b None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:02.680213 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d88c0373-44e7-4746-b430-894827e0e73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:02.853520 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095229c-7adb-4014-9491-f0f1eafb4c0b service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:03.194903 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c0373-44e7-4746-b430-894827e0e73b None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:03.222708 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d25781-dd20-4d17-b234-f5957b12e283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:03.224372 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d25781-dd20-4d17-b234-f5957b12e283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:03.224865 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d25781-dd20-4d17-b234-f5957b12e283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:03.225427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d25781-dd20-4d17-b234-f5957b12e283 admin admin] PATH_INFO: `/v3/projects/4f5c8a2e9aa64ccebb84e0a557d15e23` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:03.235113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d25781-dd20-4d17-b234-f5957b12e283 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f5c8a2e9aa64ccebb84e0a557d15e23)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:03.253973 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d25781-dd20-4d17-b234-f5957b12e283 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:03.465126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:03.566402 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-57d25781-dd20-4d17-b234-f5957b12e283 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f5c8a2e9aa64ccebb84e0a557d15e23', 'request_id': 'req-57d25781-dd20-4d17-b234-f5957b12e283'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:03.609363 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095229c-7adb-4014-9491-f0f1eafb4c0b service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:03.624076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-57d25781-dd20-4d17-b234-f5957b12e283 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f5c8a2e9aa64ccebb84e0a557d15e23', 'request_id': 'req-57d25781-dd20-4d17-b234-f5957b12e283'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:03.700016 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-57d25781-dd20-4d17-b234-f5957b12e283 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f5c8a2e9aa64ccebb84e0a557d15e23', 'request_id': 'req-57d25781-dd20-4d17-b234-f5957b12e283'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:03.886832 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-57d25781-dd20-4d17-b234-f5957b12e283 admin admin] The token cache is being invalidate because project 4f5c8a2e9aa64ccebb84e0a557d15e23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:03.890675 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-57d25781-dd20-4d17-b234-f5957b12e283 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:03.907005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-57d25781-dd20-4d17-b234-f5957b12e283 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:03.915923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 873/1742] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:02 2022] DELETE /identity/v3/projects/4f5c8a2e9aa64ccebb84e0a557d15e23 => generated 0 bytes in 1239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:31:03.950790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d25781-dd20-4d17-b234-f5957b12e283 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:03.966423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-57d25781-dd20-4d17-b234-f5957b12e283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:04.140396 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:04.238680 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df982d-bfc0-428e-9bbc-71a499d925c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:04.240544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df982d-bfc0-428e-9bbc-71a499d925c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:04.241025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df982d-bfc0-428e-9bbc-71a499d925c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:04.269699 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df982d-bfc0-428e-9bbc-71a499d925c8 admin admin] PATH_INFO: `/v3/users/161885bb40814d0da588589d2c45e8c2` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:04.280714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df982d-bfc0-428e-9bbc-71a499d925c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=161885bb40814d0da588589d2c45e8c2)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:04.284870 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df982d-bfc0-428e-9bbc-71a499d925c8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:04.698108 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a4df982d-bfc0-428e-9bbc-71a499d925c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '161885bb40814d0da588589d2c45e8c2'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:04.768397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d25781-dd20-4d17-b234-f5957b12e283 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:04.790455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a4df982d-bfc0-428e-9bbc-71a499d925c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '161885bb40814d0da588589d2c45e8c2'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:04.839083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c2f47-181b-4663-9d92-8c245b9ab524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:04.847634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c2f47-181b-4663-9d92-8c245b9ab524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:04.851694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c2f47-181b-4663-9d92-8c245b9ab524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:04.852387 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c2f47-181b-4663-9d92-8c245b9ab524 admin admin] PATH_INFO: `/v3/users/0396b085da804b6099905d150d722bf6` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:04.867883 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a4df982d-bfc0-428e-9bbc-71a499d925c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '161885bb40814d0da588589d2c45e8c2'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:04.869001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c2f47-181b-4663-9d92-8c245b9ab524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0396b085da804b6099905d150d722bf6)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:04.874258 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a4df982d-bfc0-428e-9bbc-71a499d925c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '161885bb40814d0da588589d2c45e8c2'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:04.880932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c2f47-181b-4663-9d92-8c245b9ab524 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:04.891121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a4df982d-bfc0-428e-9bbc-71a499d925c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '161885bb40814d0da588589d2c45e8c2'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:05.045182 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 870/1743] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:02 2022] DELETE /identity/v3/users/161885bb40814d0da588589d2c45e8c2 => generated 0 bytes in 2188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:05.047718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6141fd-9cad-461f-b1aa-8362af2769e3 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:05.048262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6141fd-9cad-461f-b1aa-8362af2769e3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:05.048778 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6141fd-9cad-461f-b1aa-8362af2769e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:05.233906 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3d6141fd-9cad-461f-b1aa-8362af2769e3 None None] MFA Rules not processed for user `cb2bf8531dfe42b29b0b632b39d49625`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:05.237727 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9a6c2f47-181b-4663-9d92-8c245b9ab524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0396b085da804b6099905d150d722bf6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:05.239374 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9a6c2f47-181b-4663-9d92-8c245b9ab524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0396b085da804b6099905d150d722bf6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:05.243440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3d6141fd-9cad-461f-b1aa-8362af2769e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:05.363668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9a6c2f47-181b-4663-9d92-8c245b9ab524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0396b085da804b6099905d150d722bf6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:05.375687 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9a6c2f47-181b-4663-9d92-8c245b9ab524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0396b085da804b6099905d150d722bf6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:05.405842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9a6c2f47-181b-4663-9d92-8c245b9ab524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0396b085da804b6099905d150d722bf6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:05.462085 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 874/1744] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:03 2022] DELETE /identity/v3/users/0396b085da804b6099905d150d722bf6 => generated 0 bytes in 1517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:05.464061 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb7ba9-116f-49dc-b4b2-699ad5569922 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:05.464393 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb7ba9-116f-49dc-b4b2-699ad5569922 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:05.464669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb7ba9-116f-49dc-b4b2-699ad5569922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:05.530735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-60eb7ba9-116f-49dc-b4b2-699ad5569922 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:05.533192 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-60eb7ba9-116f-49dc-b4b2-699ad5569922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:05.619945 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:05.683579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 871/1745] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 633 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:05.708090 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6141fd-9cad-461f-b1aa-8362af2769e3 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:05.711658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3d6141fd-9cad-461f-b1aa-8362af2769e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:05.765203 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 875/1746] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:05.782724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb7ba9-116f-49dc-b4b2-699ad5569922 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:05.784547 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-60eb7ba9-116f-49dc-b4b2-699ad5569922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:05.946369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb7ba9-116f-49dc-b4b2-699ad5569922 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:05.954135 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e47f3-e313-4db2-9492-529cb7e3e982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:05.954694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e47f3-e313-4db2-9492-529cb7e3e982 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:05.955140 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e47f3-e313-4db2-9492-529cb7e3e982 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:05.955433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e47f3-e313-4db2-9492-529cb7e3e982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:05.986531 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e47f3-e313-4db2-9492-529cb7e3e982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:05.988766 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e47f3-e313-4db2-9492-529cb7e3e982 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:06.022111 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 876/1747] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:06.237101 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:06.316670 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30509076-ec9a-4650-bd0f-f89b98be3fb6 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:06.324104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30509076-ec9a-4650-bd0f-f89b98be3fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:06.324104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30509076-ec9a-4650-bd0f-f89b98be3fb6 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:06.338410 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 877/1748] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 06:31:06 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:31:06.457313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97359516-e2e2-4fab-9b64-fca6da15a210 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:06.457980 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97359516-e2e2-4fab-9b64-fca6da15a210 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:06.459483 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97359516-e2e2-4fab-9b64-fca6da15a210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:06.506958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6141fd-9cad-461f-b1aa-8362af2769e3 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:06.556060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-97359516-e2e2-4fab-9b64-fca6da15a210 None None] MFA Rules not processed for user `2a09f4878eac4d60a9a7dcedba15c05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:06.561858 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-97359516-e2e2-4fab-9b64-fca6da15a210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:06.908528 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43089e03-df44-4490-90ff-bdaf6202c6cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:06.910714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43089e03-df44-4490-90ff-bdaf6202c6cc service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:06.914127 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43089e03-df44-4490-90ff-bdaf6202c6cc service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:06.914801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43089e03-df44-4490-90ff-bdaf6202c6cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:06.961541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 878/1749] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 06:31:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 507 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:07.177447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43089e03-df44-4490-90ff-bdaf6202c6cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:07.180973 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43089e03-df44-4490-90ff-bdaf6202c6cc service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:07.206203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 872/1750] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 1499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:07.596112 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97359516-e2e2-4fab-9b64-fca6da15a210 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:07.602846 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-97359516-e2e2-4fab-9b64-fca6da15a210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:07.638161 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97359516-e2e2-4fab-9b64-fca6da15a210 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:07.654441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f314a-80e1-424e-a404-ad647cf43a52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:07.655700 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f314a-80e1-424e-a404-ad647cf43a52 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:07.656001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f314a-80e1-424e-a404-ad647cf43a52 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:07.658193 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f314a-80e1-424e-a404-ad647cf43a52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:07.702299 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f314a-80e1-424e-a404-ad647cf43a52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:07.719710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f314a-80e1-424e-a404-ad647cf43a52 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:07.776636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43089e03-df44-4490-90ff-bdaf6202c6cc service nova] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:07.778972 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-43089e03-df44-4490-90ff-bdaf6202c6cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:07.807711 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 879/1751] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:07.858735 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43089e03-df44-4490-90ff-bdaf6202c6cc service nova] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:07.881006 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e321676-3659-49f7-ad02-6ee8b141a886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:07.886732 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e321676-3659-49f7-ad02-6ee8b141a886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:07.887830 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e321676-3659-49f7-ad02-6ee8b141a886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:07.889536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e321676-3659-49f7-ad02-6ee8b141a886 admin admin] PATH_INFO: `/v3/projects/d8c43c7566ba444dbc0e63a9f6c7df71` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:07.894560 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e321676-3659-49f7-ad02-6ee8b141a886 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8c43c7566ba444dbc0e63a9f6c7df71)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:07.903566 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e321676-3659-49f7-ad02-6ee8b141a886 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:08.027690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0e321676-3659-49f7-ad02-6ee8b141a886 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8c43c7566ba444dbc0e63a9f6c7df71', 'request_id': 'req-0e321676-3659-49f7-ad02-6ee8b141a886'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:08.050167 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0e321676-3659-49f7-ad02-6ee8b141a886 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8c43c7566ba444dbc0e63a9f6c7df71', 'request_id': 'req-0e321676-3659-49f7-ad02-6ee8b141a886'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:08.071048 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0e321676-3659-49f7-ad02-6ee8b141a886 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8c43c7566ba444dbc0e63a9f6c7df71', 'request_id': 'req-0e321676-3659-49f7-ad02-6ee8b141a886'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:08.124897 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0e321676-3659-49f7-ad02-6ee8b141a886 admin admin] The token cache is being invalidate because project d8c43c7566ba444dbc0e63a9f6c7df71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:08.126129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0e321676-3659-49f7-ad02-6ee8b141a886 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:08.130948 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0e321676-3659-49f7-ad02-6ee8b141a886 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:08.142792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35789da2-89a3-41ec-b0ed-1058f8733bac None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:08.143466 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35789da2-89a3-41ec-b0ed-1058f8733bac None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:08.144479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 873/1752] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:07 2022] DELETE /identity/v3/projects/d8c43c7566ba444dbc0e63a9f6c7df71 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:08.145159 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35789da2-89a3-41ec-b0ed-1058f8733bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:08.188118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-35789da2-89a3-41ec-b0ed-1058f8733bac None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:08.190553 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-35789da2-89a3-41ec-b0ed-1058f8733bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:08.268157 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:08.300852 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 880/1753] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:08.306015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7af24f-5b12-48ee-a5e0-c76f894568a6 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:08.306684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7af24f-5b12-48ee-a5e0-c76f894568a6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:08.307178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7af24f-5b12-48ee-a5e0-c76f894568a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:08.363935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5a7af24f-5b12-48ee-a5e0-c76f894568a6 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:08.367503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5a7af24f-5b12-48ee-a5e0-c76f894568a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:08.399551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 874/1754] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:08.407387 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35789da2-89a3-41ec-b0ed-1058f8733bac None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:08.437908 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35789da2-89a3-41ec-b0ed-1058f8733bac None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:08.449355 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373be4ac-9d5f-47ce-9aae-ffc75089e560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:08.451481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373be4ac-9d5f-47ce-9aae-ffc75089e560 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:08.452274 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373be4ac-9d5f-47ce-9aae-ffc75089e560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:08.452897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373be4ac-9d5f-47ce-9aae-ffc75089e560 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:08.459923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373be4ac-9d5f-47ce-9aae-ffc75089e560 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:31:08.459923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373be4ac-9d5f-47ce-9aae-ffc75089e560 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:08.459923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373be4ac-9d5f-47ce-9aae-ffc75089e560 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:08.497184 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 881/1755] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:31:08.499057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7af24f-5b12-48ee-a5e0-c76f894568a6 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:08.549875 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b6b5d-bf9c-493b-a738-be5596da30c5 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:08.550665 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b6b5d-bf9c-493b-a738-be5596da30c5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:08.553829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b6b5d-bf9c-493b-a738-be5596da30c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:08.556418 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7af24f-5b12-48ee-a5e0-c76f894568a6 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:08.592560 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5889b1-db4f-470d-8b08-3a882d9ca4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:08.599423 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5889b1-db4f-470d-8b08-3a882d9ca4fe admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:08.599423 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5889b1-db4f-470d-8b08-3a882d9ca4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:08.599423 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5889b1-db4f-470d-8b08-3a882d9ca4fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:08.599423 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5889b1-db4f-470d-8b08-3a882d9ca4fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:08.599423 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5889b1-db4f-470d-8b08-3a882d9ca4fe admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:08.624054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:08.679608 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 875/1756] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:08 2022] POST /identity/v3/projects => generated 379 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:08.681060 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5889b1-db4f-470d-8b08-3a882d9ca4fe admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:08.688449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ae3b6b5d-bf9c-493b-a738-be5596da30c5 None None] MFA Rules not processed for user `7a3536012d1444a1b9b606c9ec94e5cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:08.694043 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ae3b6b5d-bf9c-493b-a738-be5596da30c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:08.965367 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5889b1-db4f-470d-8b08-3a882d9ca4fe admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:08.975579 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 882/1757] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:31:08.977094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b6b5d-bf9c-493b-a738-be5596da30c5 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:08.982061 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ae3b6b5d-bf9c-493b-a738-be5596da30c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:08.997499 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ab147-0e07-4c3a-880c-3efa188d3a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:08.998209 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ab147-0e07-4c3a-880c-3efa188d3a5f admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:08.998708 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ab147-0e07-4c3a-880c-3efa188d3a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:08.999165 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ab147-0e07-4c3a-880c-3efa188d3a5f admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:09.000052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ab147-0e07-4c3a-880c-3efa188d3a5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:09.002292 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ab147-0e07-4c3a-880c-3efa188d3a5f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:09.064954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 876/1758] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:08 2022] POST /identity/v3/users => generated 290 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:09.066154 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ab147-0e07-4c3a-880c-3efa188d3a5f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:09.078369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b6b5d-bf9c-493b-a738-be5596da30c5 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:09.079906 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-926ab147-0e07-4c3a-880c-3efa188d3a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:09.111717 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe93fc-a71a-4738-bbdc-efc54b5fed37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:09.114729 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe93fc-a71a-4738-bbdc-efc54b5fed37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:09.121427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe93fc-a71a-4738-bbdc-efc54b5fed37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:09.122671 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe93fc-a71a-4738-bbdc-efc54b5fed37 admin admin] PATH_INFO: `/v3/projects/88e493edc5d946c295aab5d435a58cc4` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:09.126700 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe93fc-a71a-4738-bbdc-efc54b5fed37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e493edc5d946c295aab5d435a58cc4)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:09.135955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe93fc-a71a-4738-bbdc-efc54b5fed37 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:09.279273 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5efe93fc-a71a-4738-bbdc-efc54b5fed37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e493edc5d946c295aab5d435a58cc4', 'request_id': 'req-5efe93fc-a71a-4738-bbdc-efc54b5fed37'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:09.305732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5efe93fc-a71a-4738-bbdc-efc54b5fed37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e493edc5d946c295aab5d435a58cc4', 'request_id': 'req-5efe93fc-a71a-4738-bbdc-efc54b5fed37'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:09.329534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5efe93fc-a71a-4738-bbdc-efc54b5fed37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e493edc5d946c295aab5d435a58cc4', 'request_id': 'req-5efe93fc-a71a-4738-bbdc-efc54b5fed37'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:09.330801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ab147-0e07-4c3a-880c-3efa188d3a5f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:09.360380 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5efe93fc-a71a-4738-bbdc-efc54b5fed37 admin admin] The token cache is being invalidate because project 88e493edc5d946c295aab5d435a58cc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:09.360691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5efe93fc-a71a-4738-bbdc-efc54b5fed37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:09.361875 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5efe93fc-a71a-4738-bbdc-efc54b5fed37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:09.372049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 883/1759] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:08 2022] DELETE /identity/v3/projects/88e493edc5d946c295aab5d435a58cc4 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:09.372049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe93fc-a71a-4738-bbdc-efc54b5fed37 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:09.379906 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5efe93fc-a71a-4738-bbdc-efc54b5fed37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:09.506697 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61d8e9-a933-494c-8cb5-70e2f0db8428 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:09.507448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61d8e9-a933-494c-8cb5-70e2f0db8428 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:09.507747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61d8e9-a933-494c-8cb5-70e2f0db8428 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:09.508029 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61d8e9-a933-494c-8cb5-70e2f0db8428 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:09.512698 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ce61d8e9-a933-494c-8cb5-70e2f0db8428 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:09.540846 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:09.557707 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:09.564044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe93fc-a71a-4738-bbdc-efc54b5fed37 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:09.579278 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc9043-f0e7-440e-b662-2104ae6df4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:09.579992 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc9043-f0e7-440e-b662-2104ae6df4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:09.580507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc9043-f0e7-440e-b662-2104ae6df4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:09.580807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc9043-f0e7-440e-b662-2104ae6df4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:09.581610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc9043-f0e7-440e-b662-2104ae6df4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:09.583846 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc9043-f0e7-440e-b662-2104ae6df4d3 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:09.612181 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 884/1760] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:09.612919 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc9043-f0e7-440e-b662-2104ae6df4d3 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:09.626073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-52bc9043-f0e7-440e-b662-2104ae6df4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:09.660702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc9043-f0e7-440e-b662-2104ae6df4d3 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:09.688078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6d191-ac80-4f92-86c7-a3a28141365d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:09.689015 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6d191-ac80-4f92-86c7-a3a28141365d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:09.689322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6d191-ac80-4f92-86c7-a3a28141365d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:09.689614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6d191-ac80-4f92-86c7-a3a28141365d admin admin] PATH_INFO: `/v3/users/2fcebcd0e08a4e24adbd3e8591d61712` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:09.693067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6d191-ac80-4f92-86c7-a3a28141365d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fcebcd0e08a4e24adbd3e8591d61712)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:09.697844 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61d8e9-a933-494c-8cb5-70e2f0db8428 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:09.699587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61d8e9-a933-494c-8cb5-70e2f0db8428 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:09.703978 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6d191-ac80-4f92-86c7-a3a28141365d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:09.744211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 877/1761] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 678 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:09.745264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61d8e9-a933-494c-8cb5-70e2f0db8428 service nova] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:09.783899 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61d8e9-a933-494c-8cb5-70e2f0db8428 service nova] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:09.796562 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4a2ad-c755-4112-ad9c-176cb601aa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:09.797266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4a2ad-c755-4112-ad9c-176cb601aa3e admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:09.797685 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4a2ad-c755-4112-ad9c-176cb601aa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:09.797974 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4a2ad-c755-4112-ad9c-176cb601aa3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:09.799377 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4a2ad-c755-4112-ad9c-176cb601aa3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:09.801609 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4a2ad-c755-4112-ad9c-176cb601aa3e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:09.851179 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 878/1762] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:09.856055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4a2ad-c755-4112-ad9c-176cb601aa3e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:09.861929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c7a6d191-ac80-4f92-86c7-a3a28141365d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fcebcd0e08a4e24adbd3e8591d61712'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:09.865768 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c7a6d191-ac80-4f92-86c7-a3a28141365d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fcebcd0e08a4e24adbd3e8591d61712'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:09.895742 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4a2ad-c755-4112-ad9c-176cb601aa3e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:09.918007 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61436d13-acb4-48f8-89c3-08210f55aa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:09.919492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c7a6d191-ac80-4f92-86c7-a3a28141365d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fcebcd0e08a4e24adbd3e8591d61712'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:09.921328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61436d13-acb4-48f8-89c3-08210f55aa59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:09.922360 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c7a6d191-ac80-4f92-86c7-a3a28141365d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fcebcd0e08a4e24adbd3e8591d61712'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:09.925782 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61436d13-acb4-48f8-89c3-08210f55aa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:09.926292 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61436d13-acb4-48f8-89c3-08210f55aa59 admin admin] PATH_INFO: `/v3/projects/b2870946343b417b90c6ce9e52b081a7/users/562f107dcf24468bb86ad91adbacbd3b/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:09.960890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c7a6d191-ac80-4f92-86c7-a3a28141365d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fcebcd0e08a4e24adbd3e8591d61712'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:10.012287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61436d13-acb4-48f8-89c3-08210f55aa59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2870946343b417b90c6ce9e52b081a7, user_id=562f107dcf24468bb86ad91adbacbd3b, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:10.017909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61436d13-acb4-48f8-89c3-08210f55aa59 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:10.028089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 885/1763] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:09 2022] DELETE /identity/v3/users/2fcebcd0e08a4e24adbd3e8591d61712 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:10.050472 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 879/1764] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:09 2022] PUT /identity/v3/projects/b2870946343b417b90c6ce9e52b081a7/users/562f107dcf24468bb86ad91adbacbd3b/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:10.071267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79f865-ae78-4b34-a63e-c31f70cd8af9 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:10.071267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79f865-ae78-4b34-a63e-c31f70cd8af9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:10.071630 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79f865-ae78-4b34-a63e-c31f70cd8af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:10.167672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3d79f865-ae78-4b34-a63e-c31f70cd8af9 None None] MFA Rules not processed for user `562f107dcf24468bb86ad91adbacbd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:10.171233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3d79f865-ae78-4b34-a63e-c31f70cd8af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:10.448494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 886/1765] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:31:10.451278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61436d13-acb4-48f8-89c3-08210f55aa59 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:10.455668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-61436d13-acb4-48f8-89c3-08210f55aa59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:10.774968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98a1ea-0db8-46fc-b788-3f770ea7b6df None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:10.775537 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98a1ea-0db8-46fc-b788-3f770ea7b6df None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:10.775989 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98a1ea-0db8-46fc-b788-3f770ea7b6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:10.776118 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61436d13-acb4-48f8-89c3-08210f55aa59 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:10.796727 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d2de1-211e-4239-889e-7d850f6c41b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:10.797938 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d2de1-211e-4239-889e-7d850f6c41b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:10.798432 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d2de1-211e-4239-889e-7d850f6c41b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:10.798931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d2de1-211e-4239-889e-7d850f6c41b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:10.800292 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d2de1-211e-4239-889e-7d850f6c41b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:10.815188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d2de1-211e-4239-889e-7d850f6c41b3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:10.881208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9e98a1ea-0db8-46fc-b788-3f770ea7b6df None None] MFA Rules not processed for user `6a71c3c016564c3b9ab0b6b58d378fda`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:10.883405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9e98a1ea-0db8-46fc-b788-3f770ea7b6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:10.936421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 880/1766] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:10 2022] POST /identity/v3/projects => generated 381 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:31:10.938542 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d2de1-211e-4239-889e-7d850f6c41b3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:11.073129 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:11.183302 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d2de1-211e-4239-889e-7d850f6c41b3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:11.198587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203df15-81c1-4c88-b92d-74a01f1e1b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:11.199205 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203df15-81c1-4c88-b92d-74a01f1e1b34 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:11.199492 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203df15-81c1-4c88-b92d-74a01f1e1b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:11.199770 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203df15-81c1-4c88-b92d-74a01f1e1b34 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:11.200514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5203df15-81c1-4c88-b92d-74a01f1e1b34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:11.202773 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5203df15-81c1-4c88-b92d-74a01f1e1b34 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:11.210355 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 887/1767] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:11.228153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862adbc-e57e-4afa-a8fb-9636b0a81a42 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:11.229206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862adbc-e57e-4afa-a8fb-9636b0a81a42 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:11.230137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862adbc-e57e-4afa-a8fb-9636b0a81a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:11.230890 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:11.257239 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 881/1768] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:10 2022] POST /identity/v3/users => generated 297 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:11.258919 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203df15-81c1-4c88-b92d-74a01f1e1b34 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:11.285452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203df15-81c1-4c88-b92d-74a01f1e1b34 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:11.306628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970e305-2768-48a3-b69f-f66b52c8d789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:11.307347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970e305-2768-48a3-b69f-f66b52c8d789 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:11.307637 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970e305-2768-48a3-b69f-f66b52c8d789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:11.308055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970e305-2768-48a3-b69f-f66b52c8d789 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:11.309815 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970e305-2768-48a3-b69f-f66b52c8d789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:11.316757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970e305-2768-48a3-b69f-f66b52c8d789 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:11.339257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3862adbc-e57e-4afa-a8fb-9636b0a81a42 None None] MFA Rules not processed for user `6a71c3c016564c3b9ab0b6b58d378fda`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:11.359615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3862adbc-e57e-4afa-a8fb-9636b0a81a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:11.362856 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 882/1769] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:11.364175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970e305-2768-48a3-b69f-f66b52c8d789 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:11.390880 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970e305-2768-48a3-b69f-f66b52c8d789 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:11.409040 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ee1c2-c621-4ea9-b193-90c409fa73b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:11.410640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ee1c2-c621-4ea9-b193-90c409fa73b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:11.411717 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ee1c2-c621-4ea9-b193-90c409fa73b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:11.412755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ee1c2-c621-4ea9-b193-90c409fa73b9 admin admin] PATH_INFO: `/v3/projects/f08aa731e21048bb9ebac0dfc7bb12e1/users/db1ee7466457419a9ab688eb04a50aad/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:11.504339 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ee1c2-c621-4ea9-b193-90c409fa73b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f08aa731e21048bb9ebac0dfc7bb12e1, user_id=db1ee7466457419a9ab688eb04a50aad, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:11.515662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ee1c2-c621-4ea9-b193-90c409fa73b9 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:11.558762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 883/1770] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:11 2022] PUT /identity/v3/projects/f08aa731e21048bb9ebac0dfc7bb12e1/users/db1ee7466457419a9ab688eb04a50aad/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:11.563088 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ee1c2-c621-4ea9-b193-90c409fa73b9 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:11.625206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 888/1771] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:11.634186 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862adbc-e57e-4afa-a8fb-9636b0a81a42 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:11.636685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3862adbc-e57e-4afa-a8fb-9636b0a81a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:11.686989 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862adbc-e57e-4afa-a8fb-9636b0a81a42 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:11.728173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ee1c2-c621-4ea9-b193-90c409fa73b9 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:11.735018 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293faf51-c5be-40c7-97dd-53ee5aa483dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:11.735740 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293faf51-c5be-40c7-97dd-53ee5aa483dd admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:11.736034 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293faf51-c5be-40c7-97dd-53ee5aa483dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:11.736326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293faf51-c5be-40c7-97dd-53ee5aa483dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:11.737145 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293faf51-c5be-40c7-97dd-53ee5aa483dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:11.754491 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293faf51-c5be-40c7-97dd-53ee5aa483dd admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:11.780328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 884/1772] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:11.782148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293faf51-c5be-40c7-97dd-53ee5aa483dd admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:11.798850 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-293faf51-c5be-40c7-97dd-53ee5aa483dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:11.882154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6436a51-345a-4655-bc8a-365cb8a18b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:11.883988 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6436a51-345a-4655-bc8a-365cb8a18b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:11.884571 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6436a51-345a-4655-bc8a-365cb8a18b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:11.885216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6436a51-345a-4655-bc8a-365cb8a18b97 admin admin] PATH_INFO: `/v3/projects/478d6acaf74341739715f97ea06ba41c` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:11.889354 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6436a51-345a-4655-bc8a-365cb8a18b97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=478d6acaf74341739715f97ea06ba41c)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:11.895120 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6436a51-345a-4655-bc8a-365cb8a18b97 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:12.115008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f6436a51-345a-4655-bc8a-365cb8a18b97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '478d6acaf74341739715f97ea06ba41c', 'request_id': 'req-f6436a51-345a-4655-bc8a-365cb8a18b97'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:12.175467 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f6436a51-345a-4655-bc8a-365cb8a18b97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '478d6acaf74341739715f97ea06ba41c', 'request_id': 'req-f6436a51-345a-4655-bc8a-365cb8a18b97'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:12.190313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f6436a51-345a-4655-bc8a-365cb8a18b97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '478d6acaf74341739715f97ea06ba41c', 'request_id': 'req-f6436a51-345a-4655-bc8a-365cb8a18b97'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:12.201726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293faf51-c5be-40c7-97dd-53ee5aa483dd admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:12.229608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f6436a51-345a-4655-bc8a-365cb8a18b97 admin admin] The token cache is being invalidate because project 478d6acaf74341739715f97ea06ba41c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:12.230032 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f6436a51-345a-4655-bc8a-365cb8a18b97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:12.233920 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f6436a51-345a-4655-bc8a-365cb8a18b97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:12.246151 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 889/1773] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:11 2022] DELETE /identity/v3/projects/478d6acaf74341739715f97ea06ba41c => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:12.246151 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6436a51-345a-4655-bc8a-365cb8a18b97 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:12.254846 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f6436a51-345a-4655-bc8a-365cb8a18b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:12.355703 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:12.552689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6436a51-345a-4655-bc8a-365cb8a18b97 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:12.562500 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:12.586104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f481e-d1cb-4987-a312-b6506e233f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:12.587515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f481e-d1cb-4987-a312-b6506e233f27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:12.587811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f481e-d1cb-4987-a312-b6506e233f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:12.588090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f481e-d1cb-4987-a312-b6506e233f27 admin admin] PATH_INFO: `/v3/projects/f08aa731e21048bb9ebac0dfc7bb12e1/users/db1ee7466457419a9ab688eb04a50aad/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:12.594487 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f481e-d1cb-4987-a312-b6506e233f27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f08aa731e21048bb9ebac0dfc7bb12e1, user_id=db1ee7466457419a9ab688eb04a50aad, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:12.597872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f481e-d1cb-4987-a312-b6506e233f27 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:12.639088 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf56546-282f-4f14-811a-8e6d2694bab7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:12.640252 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf56546-282f-4f14-811a-8e6d2694bab7 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:12.651952 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 890/1774] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:12 2022] PUT /identity/v3/projects/f08aa731e21048bb9ebac0dfc7bb12e1/users/db1ee7466457419a9ab688eb04a50aad/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:12.651952 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d627e9-cb70-49d6-b8cc-d5f2e221197c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:12.651952 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d627e9-cb70-49d6-b8cc-d5f2e221197c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:12.651952 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d627e9-cb70-49d6-b8cc-d5f2e221197c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:12.655533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf56546-282f-4f14-811a-8e6d2694bab7 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:12.659764 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf56546-282f-4f14-811a-8e6d2694bab7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:12.675200 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cbf56546-282f-4f14-811a-8e6d2694bab7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:12.695894 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f3d627e9-cb70-49d6-b8cc-d5f2e221197c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:12.699084 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f3d627e9-cb70-49d6-b8cc-d5f2e221197c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:12.979268 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf56546-282f-4f14-811a-8e6d2694bab7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:12.983110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf56546-282f-4f14-811a-8e6d2694bab7 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:13.056516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 885/1775] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 1275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:13.059565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b021d-8ad1-4e25-9c6d-e2e1fd786a65 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:13.061662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b021d-8ad1-4e25-9c6d-e2e1fd786a65 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:13.063153 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b021d-8ad1-4e25-9c6d-e2e1fd786a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:13.066666 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 891/1776] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:13.075432 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ade67-b51d-4b65-b7cc-c1da936fc842 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:13.077152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ade67-b51d-4b65-b7cc-c1da936fc842 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:13.078538 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ade67-b51d-4b65-b7cc-c1da936fc842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:13.174536 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d94ade67-b51d-4b65-b7cc-c1da936fc842 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:13.183011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d94ade67-b51d-4b65-b7cc-c1da936fc842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:13.196606 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9d6b021d-8ad1-4e25-9c6d-e2e1fd786a65 None None] MFA Rules not processed for user `db1ee7466457419a9ab688eb04a50aad`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:13.201192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9d6b021d-8ad1-4e25-9c6d-e2e1fd786a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:13.252503 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 892/1777] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:13.258594 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ade67-b51d-4b65-b7cc-c1da936fc842 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:13.317371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ade67-b51d-4b65-b7cc-c1da936fc842 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:13.374528 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eafde4-15f8-41f1-95a2-cfcaa03c6df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:13.379511 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eafde4-15f8-41f1-95a2-cfcaa03c6df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:13.382096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eafde4-15f8-41f1-95a2-cfcaa03c6df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:13.382466 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eafde4-15f8-41f1-95a2-cfcaa03c6df9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:13.385600 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eafde4-15f8-41f1-95a2-cfcaa03c6df9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:31:13.385883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eafde4-15f8-41f1-95a2-cfcaa03c6df9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:13.387896 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eafde4-15f8-41f1-95a2-cfcaa03c6df9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:13.457391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 893/1778] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:31:13.467059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eafde4-15f8-41f1-95a2-cfcaa03c6df9 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:13.490205 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eafde4-15f8-41f1-95a2-cfcaa03c6df9 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:13.512279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e129549-fc35-44c9-9750-108b6c3f43ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:13.514557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e129549-fc35-44c9-9750-108b6c3f43ed admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:13.515872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e129549-fc35-44c9-9750-108b6c3f43ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:13.516158 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e129549-fc35-44c9-9750-108b6c3f43ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:13.516998 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e129549-fc35-44c9-9750-108b6c3f43ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:13.524983 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e129549-fc35-44c9-9750-108b6c3f43ed admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:13.576870 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 886/1779] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 520 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:13.598906 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b021d-8ad1-4e25-9c6d-e2e1fd786a65 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:13.606046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9d6b021d-8ad1-4e25-9c6d-e2e1fd786a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:13.637136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 894/1780] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:13 2022] POST /identity/v3/projects => generated 387 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:13.644763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e129549-fc35-44c9-9750-108b6c3f43ed admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:13.923365 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e129549-fc35-44c9-9750-108b6c3f43ed admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:13.939365 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6ced9-1d8e-4cf0-bc83-d9e2def7028c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:13.940655 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6ced9-1d8e-4cf0-bc83-d9e2def7028c admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:13.943397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6ced9-1d8e-4cf0-bc83-d9e2def7028c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:13.945311 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6ced9-1d8e-4cf0-bc83-d9e2def7028c admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:13.949390 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6ced9-1d8e-4cf0-bc83-d9e2def7028c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:13.953663 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6ced9-1d8e-4cf0-bc83-d9e2def7028c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:13.994747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b021d-8ad1-4e25-9c6d-e2e1fd786a65 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:14.000550 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:14.034974 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 895/1781] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:13 2022] POST /identity/v3/users => generated 294 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:14.041970 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6ced9-1d8e-4cf0-bc83-d9e2def7028c admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:14.087686 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6ced9-1d8e-4cf0-bc83-d9e2def7028c admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:14.115031 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bab99-4d3a-4fcd-a99b-9b575c5f03ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:14.119984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bab99-4d3a-4fcd-a99b-9b575c5f03ac admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:14.124877 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bab99-4d3a-4fcd-a99b-9b575c5f03ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:14.126205 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bab99-4d3a-4fcd-a99b-9b575c5f03ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:14.130176 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580bab99-4d3a-4fcd-a99b-9b575c5f03ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:14.137932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580bab99-4d3a-4fcd-a99b-9b575c5f03ac admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:14.191242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:14.199073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 896/1782] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:14.204859 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bab99-4d3a-4fcd-a99b-9b575c5f03ac admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:14.244082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bab99-4d3a-4fcd-a99b-9b575c5f03ac admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:14.279238 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2dfb8-a593-4eeb-ac9b-5b2d7c197b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:14.280955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2dfb8-a593-4eeb-ac9b-5b2d7c197b38 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:14.283084 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2dfb8-a593-4eeb-ac9b-5b2d7c197b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:14.285567 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2dfb8-a593-4eeb-ac9b-5b2d7c197b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:14.291622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2dfb8-a593-4eeb-ac9b-5b2d7c197b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:14.297692 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad0027-8648-47e8-9f83-879af61b2161 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:14.298499 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad0027-8648-47e8-9f83-879af61b2161 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:14.298883 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad0027-8648-47e8-9f83-879af61b2161 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:14.299242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad0027-8648-47e8-9f83-879af61b2161 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:14.301593 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2dfb8-a593-4eeb-ac9b-5b2d7c197b38 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:14.392054 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 897/1783] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:31:14.399883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2dfb8-a593-4eeb-ac9b-5b2d7c197b38 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:14.438550 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2dfb8-a593-4eeb-ac9b-5b2d7c197b38 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:14.453265 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0700670-d090-4122-aac7-ae9c0e5f7e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:14.463409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0700670-d090-4122-aac7-ae9c0e5f7e8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:14.463800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0700670-d090-4122-aac7-ae9c0e5f7e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:14.465196 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0700670-d090-4122-aac7-ae9c0e5f7e8b admin admin] PATH_INFO: `/v3/projects/f25221589bf04cdaaaa799af6f2bfd77/users/32c3249883a4401abe8c1941febe7cb2/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:14.525522 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0700670-d090-4122-aac7-ae9c0e5f7e8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f25221589bf04cdaaaa799af6f2bfd77, user_id=32c3249883a4401abe8c1941febe7cb2, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:14.531173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad0027-8648-47e8-9f83-879af61b2161 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:14.531173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad0027-8648-47e8-9f83-879af61b2161 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:14.578360 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 887/1784] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 983 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:14.602184 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0700670-d090-4122-aac7-ae9c0e5f7e8b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:14.671563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 898/1785] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:14 2022] PUT /identity/v3/projects/f25221589bf04cdaaaa799af6f2bfd77/users/32c3249883a4401abe8c1941febe7cb2/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:14.695281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a407c0-1730-45be-9830-50c86334b7db None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:14.695806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a407c0-1730-45be-9830-50c86334b7db None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:14.696317 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a407c0-1730-45be-9830-50c86334b7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:14.841531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-58a407c0-1730-45be-9830-50c86334b7db None None] MFA Rules not processed for user `32c3249883a4401abe8c1941febe7cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:14.850588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-58a407c0-1730-45be-9830-50c86334b7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:15.146646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 888/1786] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:31:15.148596 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b370dd4-091e-4e41-9827-4c6600c40149 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:15.149334 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b370dd4-091e-4e41-9827-4c6600c40149 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:15.149334 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b370dd4-091e-4e41-9827-4c6600c40149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:15.168293 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a407c0-1730-45be-9830-50c86334b7db None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:15.197448 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-1b370dd4-091e-4e41-9827-4c6600c40149 None None] MFA Rules not processed for user `cb2bf8531dfe42b29b0b632b39d49625`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:15.199993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1b370dd4-091e-4e41-9827-4c6600c40149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:15.473701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a407c0-1730-45be-9830-50c86334b7db None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:15.498435 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672b13d-219a-40ff-bc86-e56db6e6e806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:15.498435 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672b13d-219a-40ff-bc86-e56db6e6e806 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:15.498435 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672b13d-219a-40ff-bc86-e56db6e6e806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:15.498435 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672b13d-219a-40ff-bc86-e56db6e6e806 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:15.498860 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672b13d-219a-40ff-bc86-e56db6e6e806 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:15.498935 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 899/1787] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:15.499546 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672b13d-219a-40ff-bc86-e56db6e6e806 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:15.549476 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:15.569845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 889/1788] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:15 2022] POST /identity/v3/projects => generated 385 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:15.574552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b370dd4-091e-4e41-9827-4c6600c40149 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:15.598070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b370dd4-091e-4e41-9827-4c6600c40149 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:15.819562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c89ba-05e9-47b5-b096-7ccbf7e767e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:15.820468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c89ba-05e9-47b5-b096-7ccbf7e767e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:15.826996 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c89ba-05e9-47b5-b096-7ccbf7e767e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:15.827611 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c89ba-05e9-47b5-b096-7ccbf7e767e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:15.828599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c89ba-05e9-47b5-b096-7ccbf7e767e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:15.831047 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c89ba-05e9-47b5-b096-7ccbf7e767e0 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:15.928703 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 900/1789] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:15 2022] POST /identity/v3/users => generated 293 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:15.935725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672b13d-219a-40ff-bc86-e56db6e6e806 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:15.964272 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672b13d-219a-40ff-bc86-e56db6e6e806 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:15.977489 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c89ba-05e9-47b5-b096-7ccbf7e767e0 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:16.002273 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af1e3e-b4e8-4f6f-871a-70ef5b31f064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:16.002989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af1e3e-b4e8-4f6f-871a-70ef5b31f064 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:16.003278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af1e3e-b4e8-4f6f-871a-70ef5b31f064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:16.003559 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af1e3e-b4e8-4f6f-871a-70ef5b31f064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:16.004312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af1e3e-b4e8-4f6f-871a-70ef5b31f064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:16.019477 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af1e3e-b4e8-4f6f-871a-70ef5b31f064 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:16.066818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 890/1790] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:16.072533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af1e3e-b4e8-4f6f-871a-70ef5b31f064 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:16.098641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af1e3e-b4e8-4f6f-871a-70ef5b31f064 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:16.118385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67a7db-d93a-4423-87d0-84ffe26650c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:16.120248 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67a7db-d93a-4423-87d0-84ffe26650c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:16.123262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67a7db-d93a-4423-87d0-84ffe26650c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:16.123782 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67a7db-d93a-4423-87d0-84ffe26650c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:16.125607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67a7db-d93a-4423-87d0-84ffe26650c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:16.129669 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67a7db-d93a-4423-87d0-84ffe26650c5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:16.137335 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:16.164356 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 891/1791] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:16.169722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67a7db-d93a-4423-87d0-84ffe26650c5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:16.174287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c89ba-05e9-47b5-b096-7ccbf7e767e0 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:16.189006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4db81-be60-48b6-a2bf-8cf306918c15 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1ee7466457419a9ab688eb04a50aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f08aa731e21048bb9ebac0dfc7bb12e1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:16.191880 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67a7db-d93a-4423-87d0-84ffe26650c5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:16.193526 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4db81-be60-48b6-a2bf-8cf306918c15 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:16.194639 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4db81-be60-48b6-a2bf-8cf306918c15 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:16.195090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4db81-be60-48b6-a2bf-8cf306918c15 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:16.198667 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4db81-be60-48b6-a2bf-8cf306918c15 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:31:16.199562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4db81-be60-48b6-a2bf-8cf306918c15 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:16.205092 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4db81-be60-48b6-a2bf-8cf306918c15 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:16.213724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2253dad-396f-4aa6-801c-65f33761139f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:16.214768 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2253dad-396f-4aa6-801c-65f33761139f admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:16.215066 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2253dad-396f-4aa6-801c-65f33761139f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:16.216456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2253dad-396f-4aa6-801c-65f33761139f admin admin] PATH_INFO: `/v3/projects/2a97e366c04345ecb93236454316b2fb/users/7cced47d0020445b9ce5eae9aa908d55/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:16.242149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 901/1792] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:31:16.242888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4db81-be60-48b6-a2bf-8cf306918c15 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:16.248838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-73d4db81-be60-48b6-a2bf-8cf306918c15 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:16.259755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2253dad-396f-4aa6-801c-65f33761139f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a97e366c04345ecb93236454316b2fb, user_id=7cced47d0020445b9ce5eae9aa908d55, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:16.268608 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2253dad-396f-4aa6-801c-65f33761139f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:16.286257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4db81-be60-48b6-a2bf-8cf306918c15 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:16.306540 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 892/1793] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:16 2022] PUT /identity/v3/projects/2a97e366c04345ecb93236454316b2fb/users/7cced47d0020445b9ce5eae9aa908d55/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:16.308624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2253dad-396f-4aa6-801c-65f33761139f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:16.453429 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b612768-2b1c-4887-83e0-4b494cf81b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:16.454370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b612768-2b1c-4887-83e0-4b494cf81b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:16.454676 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b612768-2b1c-4887-83e0-4b494cf81b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:16.454946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b612768-2b1c-4887-83e0-4b494cf81b28 admin admin] PATH_INFO: `/v3/users/a1dd172493724e959fdb915c6dc6b214` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:16.467772 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b612768-2b1c-4887-83e0-4b494cf81b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1dd172493724e959fdb915c6dc6b214)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:16.469920 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b612768-2b1c-4887-83e0-4b494cf81b28 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:16.487845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2253dad-396f-4aa6-801c-65f33761139f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:16.512049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109b1f8-6a23-4a6f-b7a1-ff051b39c48b tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1ee7466457419a9ab688eb04a50aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f08aa731e21048bb9ebac0dfc7bb12e1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:16.513219 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109b1f8-6a23-4a6f-b7a1-ff051b39c48b tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:16.513744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109b1f8-6a23-4a6f-b7a1-ff051b39c48b tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:16.514278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109b1f8-6a23-4a6f-b7a1-ff051b39c48b tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:16.515527 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109b1f8-6a23-4a6f-b7a1-ff051b39c48b tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:16.524486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109b1f8-6a23-4a6f-b7a1-ff051b39c48b tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:16.633979 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2b612768-2b1c-4887-83e0-4b494cf81b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1dd172493724e959fdb915c6dc6b214'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:16.636388 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2b612768-2b1c-4887-83e0-4b494cf81b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1dd172493724e959fdb915c6dc6b214'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:16.668031 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 893/1794] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:16 2022] POST /identity/v3/projects => generated 354 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:16.668031 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9087d-5a19-48a8-a950-20f8c691bdf2 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:16.674432 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9087d-5a19-48a8-a950-20f8c691bdf2 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:16.674941 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9087d-5a19-48a8-a950-20f8c691bdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:16.692170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2b612768-2b1c-4887-83e0-4b494cf81b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1dd172493724e959fdb915c6dc6b214'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:16.694811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2b612768-2b1c-4887-83e0-4b494cf81b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1dd172493724e959fdb915c6dc6b214'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:16.728659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2b612768-2b1c-4887-83e0-4b494cf81b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1dd172493724e959fdb915c6dc6b214'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:16.747415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-25c9087d-5a19-48a8-a950-20f8c691bdf2 None None] MFA Rules not processed for user `7cced47d0020445b9ce5eae9aa908d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:16.749686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-25c9087d-5a19-48a8-a950-20f8c691bdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:16.786320 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 902/1795] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:16 2022] DELETE /identity/v3/users/a1dd172493724e959fdb915c6dc6b214 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:16.787391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b612768-2b1c-4887-83e0-4b494cf81b28 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:16.805118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2b612768-2b1c-4887-83e0-4b494cf81b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:17.040217 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 894/1796] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:17.046457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae2c3b-1aac-4eb5-8d5f-6ff8cf9d0c89 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:17.047205 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae2c3b-1aac-4eb5-8d5f-6ff8cf9d0c89 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:17.050249 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae2c3b-1aac-4eb5-8d5f-6ff8cf9d0c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:17.145588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e5ae2c3b-1aac-4eb5-8d5f-6ff8cf9d0c89 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:17.153624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e5ae2c3b-1aac-4eb5-8d5f-6ff8cf9d0c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:17.253027 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:17.288688 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b612768-2b1c-4887-83e0-4b494cf81b28 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:17.344710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c70a1f-8d65-4de1-9810-6cfc08905572 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1ee7466457419a9ab688eb04a50aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f08aa731e21048bb9ebac0dfc7bb12e1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:17.357947 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c70a1f-8d65-4de1-9810-6cfc08905572 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:17.358486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c70a1f-8d65-4de1-9810-6cfc08905572 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:17.362391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c70a1f-8d65-4de1-9810-6cfc08905572 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:17.364001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c70a1f-8d65-4de1-9810-6cfc08905572 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:31:17.364562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c70a1f-8d65-4de1-9810-6cfc08905572 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:17.372297 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c70a1f-8d65-4de1-9810-6cfc08905572 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:17.387732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:17.464070 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 895/1797] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:17.470600 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bb865-4155-44e5-b30a-f925233313f9 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:17.470888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 903/1798] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 684 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:31:17.472531 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c70a1f-8d65-4de1-9810-6cfc08905572 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:17.477370 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bb865-4155-44e5-b30a-f925233313f9 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:17.477859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bb865-4155-44e5-b30a-f925233313f9 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:17.482883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-19c70a1f-8d65-4de1-9810-6cfc08905572 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:17.500644 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 896/1799] 10.222.0.65 () {54 vars in 912 bytes} [Thu May 19 06:31:17 2022] GET /identity => generated 270 bytes in 27 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:31:17.512656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc158b85-beb6-4f8a-9db6-bf0ca0b82a7f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:17.512656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc158b85-beb6-4f8a-9db6-bf0ca0b82a7f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:17.512656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc158b85-beb6-4f8a-9db6-bf0ca0b82a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:17.539314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c70a1f-8d65-4de1-9810-6cfc08905572 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:17.560667 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb6903-be6a-44cc-bff5-45fbc4f017e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:17.562553 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb6903-be6a-44cc-bff5-45fbc4f017e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:17.563678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb6903-be6a-44cc-bff5-45fbc4f017e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:17.564897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb6903-be6a-44cc-bff5-45fbc4f017e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:17.567405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb6903-be6a-44cc-bff5-45fbc4f017e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:17.573740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb6903-be6a-44cc-bff5-45fbc4f017e2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:17.640581 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-fc158b85-beb6-4f8a-9db6-bf0ca0b82a7f None None] MFA Rules not processed for user `226f4700a516415cbbd272fdfad7af58`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:17.645116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fc158b85-beb6-4f8a-9db6-bf0ca0b82a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:17.720369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 904/1800] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:17 2022] POST /identity/v3/projects => generated 387 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:17.720369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb6903-be6a-44cc-bff5-45fbc4f017e2 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:17.720369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3edb6903-be6a-44cc-bff5-45fbc4f017e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:17.883066 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 897/1801] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 06:31:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:31:17.890088 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc158b85-beb6-4f8a-9db6-bf0ca0b82a7f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:17.927046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc158b85-beb6-4f8a-9db6-bf0ca0b82a7f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:17.960812 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb6903-be6a-44cc-bff5-45fbc4f017e2 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:17.986709 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d091f-c1b0-406b-9b1e-6753a019e345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:17.987445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d091f-c1b0-406b-9b1e-6753a019e345 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:17.987737 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d091f-c1b0-406b-9b1e-6753a019e345 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:18.009514 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d091f-c1b0-406b-9b1e-6753a019e345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:18.097259 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1ee7466457419a9ab688eb04a50aad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f08aa731e21048bb9ebac0dfc7bb12e1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:18.099146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:18.099146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:18.099146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] PATH_INFO: `/v3/projects/72bcd7acd2f5442bb3c76693168d2c4b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:18.101528 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] RBAC: Authorizing `identity:delete_project(project_id=72bcd7acd2f5442bb3c76693168d2c4b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:18.111323 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:18.295625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72bcd7acd2f5442bb3c76693168d2c4b', 'request_id': 'req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:18.328434 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72bcd7acd2f5442bb3c76693168d2c4b', 'request_id': 'req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:18.341313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72bcd7acd2f5442bb3c76693168d2c4b', 'request_id': 'req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:18.369471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] The token cache is being invalidate because project 72bcd7acd2f5442bb3c76693168d2c4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:18.372929 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:18.377110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:18.384650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d091f-c1b0-406b-9b1e-6753a019e345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:18.389906 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d091f-c1b0-406b-9b1e-6753a019e345 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:18.392064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 898/1802] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:17 2022] DELETE /identity/v3/projects/72bcd7acd2f5442bb3c76693168d2c4b => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:18.393118 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:18.398499 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:18.432265 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:18.434519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 905/1803] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 718 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:18.625268 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd26d66-84bd-437e-954e-7c6b0ba7a1d7 tempest-VolumeQuotaClassesTest-1045213694 tempest-VolumeQuotaClassesTest-1045213694-project-admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:18.638534 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3cf1d-b927-469d-82c6-07954659a067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:18.641507 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3cf1d-b927-469d-82c6-07954659a067 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:18.642214 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3cf1d-b927-469d-82c6-07954659a067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:18.644830 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3cf1d-b927-469d-82c6-07954659a067 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:18.650856 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3cf1d-b927-469d-82c6-07954659a067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:18.652110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3cf1d-b927-469d-82c6-07954659a067 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:18.759661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d091f-c1b0-406b-9b1e-6753a019e345 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:18.783285 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d091f-c1b0-406b-9b1e-6753a019e345 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:18.788149 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 899/1804] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:18 2022] POST /identity/v3/users => generated 300 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:18.795995 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3cf1d-b927-469d-82c6-07954659a067 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:18.878252 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3cf1d-b927-469d-82c6-07954659a067 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:18.899096 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea00f1-a080-492c-afb6-f8addae2acfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:18.900255 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea00f1-a080-492c-afb6-f8addae2acfa admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:18.901017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea00f1-a080-492c-afb6-f8addae2acfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:18.904141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea00f1-a080-492c-afb6-f8addae2acfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:18.904141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea00f1-a080-492c-afb6-f8addae2acfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:18.907414 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea00f1-a080-492c-afb6-f8addae2acfa admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:18.973179 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 900/1805] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:18.980405 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea00f1-a080-492c-afb6-f8addae2acfa admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:19.048379 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea00f1-a080-492c-afb6-f8addae2acfa admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:19.117330 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e409efb-95e8-43c3-8be4-25260bbc09cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:19.142886 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e409efb-95e8-43c3-8be4-25260bbc09cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:19.150923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e409efb-95e8-43c3-8be4-25260bbc09cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:19.153068 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e409efb-95e8-43c3-8be4-25260bbc09cd admin admin] PATH_INFO: `/v3/projects/9fca6ca1761540e8a265abdf8373bc15/users/b16db87abb044ab294071e5a9701cb86/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:19.229334 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3d045-e19d-4e80-8468-06f286053f85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:19.230696 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3d045-e19d-4e80-8468-06f286053f85 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:19.231564 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3d045-e19d-4e80-8468-06f286053f85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:19.232632 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3d045-e19d-4e80-8468-06f286053f85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:19.246728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-80c3d045-e19d-4e80-8468-06f286053f85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:19.274656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e409efb-95e8-43c3-8be4-25260bbc09cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fca6ca1761540e8a265abdf8373bc15, user_id=b16db87abb044ab294071e5a9701cb86, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:19.301156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e409efb-95e8-43c3-8be4-25260bbc09cd admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:19.332110 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3d045-e19d-4e80-8468-06f286053f85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:19.342408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3d045-e19d-4e80-8468-06f286053f85 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:19.401646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 901/1806] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:18 2022] PUT /identity/v3/projects/9fca6ca1761540e8a265abdf8373bc15/users/b16db87abb044ab294071e5a9701cb86/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:19.401646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e409efb-95e8-43c3-8be4-25260bbc09cd admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:19.410515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8e409efb-95e8-43c3-8be4-25260bbc09cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:19.414340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 906/1807] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:19.416737 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3d045-e19d-4e80-8468-06f286053f85 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:19.494791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3d045-e19d-4e80-8468-06f286053f85 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:19.637980 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:19.767369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e409efb-95e8-43c3-8be4-25260bbc09cd admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:19.794188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c700fc-2f51-490e-a455-0d4f76d2b7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:19.796060 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c700fc-2f51-490e-a455-0d4f76d2b7d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:19.796659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c700fc-2f51-490e-a455-0d4f76d2b7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:19.797078 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c700fc-2f51-490e-a455-0d4f76d2b7d4 admin admin] PATH_INFO: `/v3/users/562f107dcf24468bb86ad91adbacbd3b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:19.805630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c700fc-2f51-490e-a455-0d4f76d2b7d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=562f107dcf24468bb86ad91adbacbd3b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:19.811574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709c962-47ba-434d-b535-f2b0436461b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:19.815158 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709c962-47ba-434d-b535-f2b0436461b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:19.816348 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c700fc-2f51-490e-a455-0d4f76d2b7d4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:19.816917 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709c962-47ba-434d-b535-f2b0436461b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:19.818458 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709c962-47ba-434d-b535-f2b0436461b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:19.824563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f709c962-47ba-434d-b535-f2b0436461b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:19.831592 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f709c962-47ba-434d-b535-f2b0436461b1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:19.873892 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 907/1808] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:19.883000 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709c962-47ba-434d-b535-f2b0436461b1 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:19.912291 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709c962-47ba-434d-b535-f2b0436461b1 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:19.936055 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29213779-b20e-4621-a504-fea09c807c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:19.937668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29213779-b20e-4621-a504-fea09c807c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:19.937668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29213779-b20e-4621-a504-fea09c807c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:19.938173 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29213779-b20e-4621-a504-fea09c807c1c admin admin] PATH_INFO: `/v3/projects/9fca6ca1761540e8a265abdf8373bc15/users/b16db87abb044ab294071e5a9701cb86/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:19.940764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29213779-b20e-4621-a504-fea09c807c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fca6ca1761540e8a265abdf8373bc15, user_id=b16db87abb044ab294071e5a9701cb86, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:19.948396 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29213779-b20e-4621-a504-fea09c807c1c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:19.982536 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 908/1809] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:19 2022] PUT /identity/v3/projects/9fca6ca1761540e8a265abdf8373bc15/users/b16db87abb044ab294071e5a9701cb86/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:19.993770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a798d9-7dcc-4785-9af5-90086a72e791 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:19.993770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a798d9-7dcc-4785-9af5-90086a72e791 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:19.993770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a798d9-7dcc-4785-9af5-90086a72e791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:20.105388 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-58c700fc-2f51-490e-a455-0d4f76d2b7d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '562f107dcf24468bb86ad91adbacbd3b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:20.149527 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-58c700fc-2f51-490e-a455-0d4f76d2b7d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '562f107dcf24468bb86ad91adbacbd3b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:20.152590 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:20.157861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-64a798d9-7dcc-4785-9af5-90086a72e791 None None] MFA Rules not processed for user `7a3536012d1444a1b9b606c9ec94e5cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:20.162236 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-64a798d9-7dcc-4785-9af5-90086a72e791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:20.173778 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-58c700fc-2f51-490e-a455-0d4f76d2b7d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '562f107dcf24468bb86ad91adbacbd3b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:20.175164 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-58c700fc-2f51-490e-a455-0d4f76d2b7d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '562f107dcf24468bb86ad91adbacbd3b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:20.177484 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-58c700fc-2f51-490e-a455-0d4f76d2b7d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '562f107dcf24468bb86ad91adbacbd3b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:20.203280 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 902/1810] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:19 2022] DELETE /identity/v3/users/562f107dcf24468bb86ad91adbacbd3b => generated 0 bytes in 800 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:20.212545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09464527-cebc-4e11-a8d6-c999cd03db15 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:20.213856 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09464527-cebc-4e11-a8d6-c999cd03db15 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:20.214149 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09464527-cebc-4e11-a8d6-c999cd03db15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:20.292197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-09464527-cebc-4e11-a8d6-c999cd03db15 None None] MFA Rules not processed for user `b16db87abb044ab294071e5a9701cb86`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:20.293531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-09464527-cebc-4e11-a8d6-c999cd03db15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:20.508978 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 909/1811] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 525 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:20.509985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2f72f-3bac-4ca5-a020-1f09545dfaf5 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:20.510234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2f72f-3bac-4ca5-a020-1f09545dfaf5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:20.511115 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2f72f-3bac-4ca5-a020-1f09545dfaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:20.617790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-7bb2f72f-3bac-4ca5-a020-1f09545dfaf5 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:20.628362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7bb2f72f-3bac-4ca5-a020-1f09545dfaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:20.632192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 903/1812] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:20.634671 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09464527-cebc-4e11-a8d6-c999cd03db15 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:20.637155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-09464527-cebc-4e11-a8d6-c999cd03db15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:20.923628 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 910/1813] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:20.924510 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2f72f-3bac-4ca5-a020-1f09545dfaf5 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:20.926097 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7bb2f72f-3bac-4ca5-a020-1f09545dfaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:20.986113 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09464527-cebc-4e11-a8d6-c999cd03db15 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:21.002935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710174bb-e1f1-4709-809a-8021f7bfbf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:21.004018 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710174bb-e1f1-4709-809a-8021f7bfbf58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:21.004313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710174bb-e1f1-4709-809a-8021f7bfbf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:21.004594 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710174bb-e1f1-4709-809a-8021f7bfbf58 admin admin] PATH_INFO: `/v3/projects/f8725989113f47fd839541e2c213ba9a` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:21.008342 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710174bb-e1f1-4709-809a-8021f7bfbf58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8725989113f47fd839541e2c213ba9a)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:21.014289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710174bb-e1f1-4709-809a-8021f7bfbf58 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:21.202663 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2f72f-3bac-4ca5-a020-1f09545dfaf5 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:21.226630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-710174bb-e1f1-4709-809a-8021f7bfbf58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8725989113f47fd839541e2c213ba9a', 'request_id': 'req-710174bb-e1f1-4709-809a-8021f7bfbf58'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:21.243469 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-710174bb-e1f1-4709-809a-8021f7bfbf58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8725989113f47fd839541e2c213ba9a', 'request_id': 'req-710174bb-e1f1-4709-809a-8021f7bfbf58'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:21.262422 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bf934-4820-41b6-9bde-e7489f70f170 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:21.263660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bf934-4820-41b6-9bde-e7489f70f170 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:21.264178 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bf934-4820-41b6-9bde-e7489f70f170 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:21.264664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bf934-4820-41b6-9bde-e7489f70f170 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:21.291061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-710174bb-e1f1-4709-809a-8021f7bfbf58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8725989113f47fd839541e2c213ba9a', 'request_id': 'req-710174bb-e1f1-4709-809a-8021f7bfbf58'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:21.292658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:21.339663 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-710174bb-e1f1-4709-809a-8021f7bfbf58 admin admin] The token cache is being invalidate because project f8725989113f47fd839541e2c213ba9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:21.339979 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-710174bb-e1f1-4709-809a-8021f7bfbf58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:21.342820 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-710174bb-e1f1-4709-809a-8021f7bfbf58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:21.351397 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 904/1814] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:20 2022] DELETE /identity/v3/projects/f8725989113f47fd839541e2c213ba9a => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:21.351397 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710174bb-e1f1-4709-809a-8021f7bfbf58 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:21.351397 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-710174bb-e1f1-4709-809a-8021f7bfbf58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:21.357995 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4bf934-4820-41b6-9bde-e7489f70f170 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:21.368143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4bf934-4820-41b6-9bde-e7489f70f170 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:21.387877 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:21.429323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 911/1815] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:21.429323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bf934-4820-41b6-9bde-e7489f70f170 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:21.503081 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bf934-4820-41b6-9bde-e7489f70f170 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:21.724091 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710174bb-e1f1-4709-809a-8021f7bfbf58 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:21.734943 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb175143-b25b-4610-a321-b9a320f7e226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:21.737115 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb175143-b25b-4610-a321-b9a320f7e226 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:21.738764 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb175143-b25b-4610-a321-b9a320f7e226 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:21.739399 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb175143-b25b-4610-a321-b9a320f7e226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:21.743514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cb175143-b25b-4610-a321-b9a320f7e226 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:22.092851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26b838-5386-4b21-b6f7-23df2d9979e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:22.101969 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26b838-5386-4b21-b6f7-23df2d9979e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:22.104453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26b838-5386-4b21-b6f7-23df2d9979e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:22.106013 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26b838-5386-4b21-b6f7-23df2d9979e0 admin admin] PATH_INFO: `/v3/users/45c1210cf3154927ad8f3d307093b684` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:22.119078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26b838-5386-4b21-b6f7-23df2d9979e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c1210cf3154927ad8f3d307093b684)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:22.140059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26b838-5386-4b21-b6f7-23df2d9979e0 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:22.358191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb175143-b25b-4610-a321-b9a320f7e226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:22.364640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb175143-b25b-4610-a321-b9a320f7e226 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:22.430448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 905/1816] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 1071 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:22.430609 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2c26b838-5386-4b21-b6f7-23df2d9979e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c1210cf3154927ad8f3d307093b684'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:22.430609 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2c26b838-5386-4b21-b6f7-23df2d9979e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c1210cf3154927ad8f3d307093b684'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:22.430915 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca4577-cf90-4a60-87a1-7b83ca16c21c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:22.430915 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca4577-cf90-4a60-87a1-7b83ca16c21c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:22.430915 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca4577-cf90-4a60-87a1-7b83ca16c21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:22.513860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2c26b838-5386-4b21-b6f7-23df2d9979e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c1210cf3154927ad8f3d307093b684'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:22.519053 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2c26b838-5386-4b21-b6f7-23df2d9979e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c1210cf3154927ad8f3d307093b684'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:22.542178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d1ca4577-cf90-4a60-87a1-7b83ca16c21c None None] MFA Rules not processed for user `6a71c3c016564c3b9ab0b6b58d378fda`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:22.549846 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d1ca4577-cf90-4a60-87a1-7b83ca16c21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:22.558088 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2c26b838-5386-4b21-b6f7-23df2d9979e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c1210cf3154927ad8f3d307093b684'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:22.655463 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 912/1817] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:21 2022] DELETE /identity/v3/users/45c1210cf3154927ad8f3d307093b684 => generated 0 bytes in 1228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:22.914166 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 906/1818] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:23.663938 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26b838-5386-4b21-b6f7-23df2d9979e0 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:23.666529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2c26b838-5386-4b21-b6f7-23df2d9979e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:23.904469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26b838-5386-4b21-b6f7-23df2d9979e0 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:23.911922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc3939-2a41-4480-b948-e3b82c2144b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:23.913622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc3939-2a41-4480-b948-e3b82c2144b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:23.914342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc3939-2a41-4480-b948-e3b82c2144b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:23.914826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc3939-2a41-4480-b948-e3b82c2144b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:23.917335 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c0bc3939-2a41-4480-b948-e3b82c2144b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:24.024395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:24.127387 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc3939-2a41-4480-b948-e3b82c2144b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:24.129169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc3939-2a41-4480-b948-e3b82c2144b5 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:24.169135 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 913/1819] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:25.449842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca4577-cf90-4a60-87a1-7b83ca16c21c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:25.452329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d1ca4577-cf90-4a60-87a1-7b83ca16c21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:25.526695 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca4577-cf90-4a60-87a1-7b83ca16c21c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:25.589603 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc3939-2a41-4480-b948-e3b82c2144b5 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:25.591180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c0bc3939-2a41-4480-b948-e3b82c2144b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:25.731527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc3939-2a41-4480-b948-e3b82c2144b5 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:25.753897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ed85d-c1c2-49cc-9457-cf708dccfd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:25.766701 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ed85d-c1c2-49cc-9457-cf708dccfd38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:25.767512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ed85d-c1c2-49cc-9457-cf708dccfd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:25.770171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ed85d-c1c2-49cc-9457-cf708dccfd38 admin admin] PATH_INFO: `/v3/projects/1925eeac247841b98efec535b1cdbe4f` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:25.773944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ed85d-c1c2-49cc-9457-cf708dccfd38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1925eeac247841b98efec535b1cdbe4f)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:25.780442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ed85d-c1c2-49cc-9457-cf708dccfd38 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:25.795985 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:25.876476 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610026a7-a2b3-4dd7-8a71-c56d5e304d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:25.883703 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610026a7-a2b3-4dd7-8a71-c56d5e304d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:25.886714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610026a7-a2b3-4dd7-8a71-c56d5e304d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:25.887721 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610026a7-a2b3-4dd7-8a71-c56d5e304d67 admin admin] PATH_INFO: `/v3/projects/b2870946343b417b90c6ce9e52b081a7` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:25.892270 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610026a7-a2b3-4dd7-8a71-c56d5e304d67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2870946343b417b90c6ce9e52b081a7)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:25.909215 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610026a7-a2b3-4dd7-8a71-c56d5e304d67 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:25.940355 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a2ed85d-c1c2-49cc-9457-cf708dccfd38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1925eeac247841b98efec535b1cdbe4f', 'request_id': 'req-2a2ed85d-c1c2-49cc-9457-cf708dccfd38'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:25.949736 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a2ed85d-c1c2-49cc-9457-cf708dccfd38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1925eeac247841b98efec535b1cdbe4f', 'request_id': 'req-2a2ed85d-c1c2-49cc-9457-cf708dccfd38'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:25.983309 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a2ed85d-c1c2-49cc-9457-cf708dccfd38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1925eeac247841b98efec535b1cdbe4f', 'request_id': 'req-2a2ed85d-c1c2-49cc-9457-cf708dccfd38'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:26.051119 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a2ed85d-c1c2-49cc-9457-cf708dccfd38 admin admin] The token cache is being invalidate because project 1925eeac247841b98efec535b1cdbe4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:26.052664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a2ed85d-c1c2-49cc-9457-cf708dccfd38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:26.055888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2a2ed85d-c1c2-49cc-9457-cf708dccfd38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:26.063491 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 914/1820] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:25 2022] DELETE /identity/v3/projects/1925eeac247841b98efec535b1cdbe4f => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:26.070374 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c2b9d-557b-4376-ad8b-66bd1a27c3c7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:26.070670 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c2b9d-557b-4376-ad8b-66bd1a27c3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:26.070953 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c2b9d-557b-4376-ad8b-66bd1a27c3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:26.151172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-610026a7-a2b3-4dd7-8a71-c56d5e304d67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2870946343b417b90c6ce9e52b081a7', 'request_id': 'req-610026a7-a2b3-4dd7-8a71-c56d5e304d67'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:26.153406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-754c2b9d-557b-4376-ad8b-66bd1a27c3c7 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:26.156885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-754c2b9d-557b-4376-ad8b-66bd1a27c3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:26.163620 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-610026a7-a2b3-4dd7-8a71-c56d5e304d67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2870946343b417b90c6ce9e52b081a7', 'request_id': 'req-610026a7-a2b3-4dd7-8a71-c56d5e304d67'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:26.197607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-610026a7-a2b3-4dd7-8a71-c56d5e304d67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2870946343b417b90c6ce9e52b081a7', 'request_id': 'req-610026a7-a2b3-4dd7-8a71-c56d5e304d67'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:26.232952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-610026a7-a2b3-4dd7-8a71-c56d5e304d67 admin admin] The token cache is being invalidate because project b2870946343b417b90c6ce9e52b081a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:26.233975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-610026a7-a2b3-4dd7-8a71-c56d5e304d67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:26.236813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-610026a7-a2b3-4dd7-8a71-c56d5e304d67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:26.242731 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 907/1821] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:25 2022] DELETE /identity/v3/projects/b2870946343b417b90c6ce9e52b081a7 => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:26.247361 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610026a7-a2b3-4dd7-8a71-c56d5e304d67 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:26.251559 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-610026a7-a2b3-4dd7-8a71-c56d5e304d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:26.428830 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 915/1822] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:26.430264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5ef22-2ddf-4f2f-ad6a-9180ce3cd717 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:26.431257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5ef22-2ddf-4f2f-ad6a-9180ce3cd717 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:26.432975 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5ef22-2ddf-4f2f-ad6a-9180ce3cd717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:26.462686 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:26.511527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-32a5ef22-2ddf-4f2f-ad6a-9180ce3cd717 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:26.514265 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-32a5ef22-2ddf-4f2f-ad6a-9180ce3cd717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:26.571149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 916/1823] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:26.572716 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5ef22-2ddf-4f2f-ad6a-9180ce3cd717 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:26.632803 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5ef22-2ddf-4f2f-ad6a-9180ce3cd717 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:26.650650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a09623-ff8f-439d-b629-90ead2838938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:26.658291 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a09623-ff8f-439d-b629-90ead2838938 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:26.661440 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a09623-ff8f-439d-b629-90ead2838938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:26.663621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a09623-ff8f-439d-b629-90ead2838938 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:26.664993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a09623-ff8f-439d-b629-90ead2838938 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:31:26.667705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a09623-ff8f-439d-b629-90ead2838938 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:26.670411 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a09623-ff8f-439d-b629-90ead2838938 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:26.675229 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610026a7-a2b3-4dd7-8a71-c56d5e304d67 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:26.711315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 917/1824] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:31:26.713959 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a09623-ff8f-439d-b629-90ead2838938 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:26.728364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c5bc4-14a1-4fa1-9dbe-34d4dd312714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:26.732193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c5bc4-14a1-4fa1-9dbe-34d4dd312714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:26.741609 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a09623-ff8f-439d-b629-90ead2838938 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:26.742351 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c5bc4-14a1-4fa1-9dbe-34d4dd312714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:26.748844 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c5bc4-14a1-4fa1-9dbe-34d4dd312714 admin admin] PATH_INFO: `/v3/users/db1ee7466457419a9ab688eb04a50aad` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:26.756574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c5bc4-14a1-4fa1-9dbe-34d4dd312714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db1ee7466457419a9ab688eb04a50aad)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:26.758096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92da4b-8332-4972-b0d7-34a1d6636680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:26.758887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92da4b-8332-4972-b0d7-34a1d6636680 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:26.759182 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92da4b-8332-4972-b0d7-34a1d6636680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:26.759465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92da4b-8332-4972-b0d7-34a1d6636680 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:26.760296 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92da4b-8332-4972-b0d7-34a1d6636680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:26.768712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92da4b-8332-4972-b0d7-34a1d6636680 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:26.783725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c5bc4-14a1-4fa1-9dbe-34d4dd312714 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:26.899926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 918/1825] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:26 2022] POST /identity/v3/projects => generated 365 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:31:26.904075 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92da4b-8332-4972-b0d7-34a1d6636680 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:27.054234 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-da3c5bc4-14a1-4fa1-9dbe-34d4dd312714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db1ee7466457419a9ab688eb04a50aad'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:27.089976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-da3c5bc4-14a1-4fa1-9dbe-34d4dd312714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db1ee7466457419a9ab688eb04a50aad'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:27.109391 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-da3c5bc4-14a1-4fa1-9dbe-34d4dd312714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db1ee7466457419a9ab688eb04a50aad'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:27.111348 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-da3c5bc4-14a1-4fa1-9dbe-34d4dd312714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db1ee7466457419a9ab688eb04a50aad'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:27.112784 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-da3c5bc4-14a1-4fa1-9dbe-34d4dd312714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db1ee7466457419a9ab688eb04a50aad'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:27.129992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:27.134636 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92da4b-8332-4972-b0d7-34a1d6636680 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:27.144840 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 908/1826] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:26 2022] DELETE /identity/v3/users/db1ee7466457419a9ab688eb04a50aad => generated 0 bytes in 896 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:31:27.147226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72b6ba-823d-417b-99e1-32bdec752356 None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:27.147541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72b6ba-823d-417b-99e1-32bdec752356 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:27.148065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72b6ba-823d-417b-99e1-32bdec752356 None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:27.151453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 909/1827] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 06:31:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:31:27.152515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced8a6c-6de6-4474-85fa-db59801cc0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:27.153167 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced8a6c-6de6-4474-85fa-db59801cc0ed admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:27.153600 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced8a6c-6de6-4474-85fa-db59801cc0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:27.153778 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced8a6c-6de6-4474-85fa-db59801cc0ed admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:27.158815 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced8a6c-6de6-4474-85fa-db59801cc0ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:27.176206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced8a6c-6de6-4474-85fa-db59801cc0ed admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:27.187159 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e5ed8-a7e5-4711-99f7-567f11b99d8c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:27.187512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e5ed8-a7e5-4711-99f7-567f11b99d8c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:27.188063 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e5ed8-a7e5-4711-99f7-567f11b99d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:27.244815 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 919/1828] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:26 2022] POST /identity/v3/users => generated 283 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:27.249018 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced8a6c-6de6-4474-85fa-db59801cc0ed admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:27.263656 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cced8a6c-6de6-4474-85fa-db59801cc0ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:27.267744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-224e5ed8-a7e5-4711-99f7-567f11b99d8c None None] MFA Rules not processed for user `2a09f4878eac4d60a9a7dcedba15c05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:27.269898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-224e5ed8-a7e5-4711-99f7-567f11b99d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:27.304260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced8a6c-6de6-4474-85fa-db59801cc0ed admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:27.444848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 910/1829] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 06:31:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:31:27.515728 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e5ed8-a7e5-4711-99f7-567f11b99d8c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:27.517904 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-224e5ed8-a7e5-4711-99f7-567f11b99d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:27.522396 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613ef43-c337-4c93-9cc1-4ad05704185a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:27.523368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613ef43-c337-4c93-9cc1-4ad05704185a admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:27.523825 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613ef43-c337-4c93-9cc1-4ad05704185a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:27.524168 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613ef43-c337-4c93-9cc1-4ad05704185a admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:27.528494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613ef43-c337-4c93-9cc1-4ad05704185a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:27.532968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613ef43-c337-4c93-9cc1-4ad05704185a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:27.547318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e5ed8-a7e5-4711-99f7-567f11b99d8c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:27.558878 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c0fdd-a025-4d57-b318-05916f9e0431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:27.560274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c0fdd-a025-4d57-b318-05916f9e0431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:27.560572 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c0fdd-a025-4d57-b318-05916f9e0431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:27.560853 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c0fdd-a025-4d57-b318-05916f9e0431 admin admin] PATH_INFO: `/v3/users/a185f5eca38344a99a5333cbac424015` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:27.562745 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c0fdd-a025-4d57-b318-05916f9e0431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a185f5eca38344a99a5333cbac424015)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:27.565105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c0fdd-a025-4d57-b318-05916f9e0431 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:27.588171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 920/1830] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:27.596452 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613ef43-c337-4c93-9cc1-4ad05704185a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:27.688550 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613ef43-c337-4c93-9cc1-4ad05704185a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:27.720117 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-f10c0fdd-a025-4d57-b318-05916f9e0431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a185f5eca38344a99a5333cbac424015'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:27.731973 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0c5ed-c48b-4227-a641-eefa6a61e407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:27.739615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0c5ed-c48b-4227-a641-eefa6a61e407 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:27.740193 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0c5ed-c48b-4227-a641-eefa6a61e407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:27.745611 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0c5ed-c48b-4227-a641-eefa6a61e407 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:27.750384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0c5ed-c48b-4227-a641-eefa6a61e407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:27.758357 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0c5ed-c48b-4227-a641-eefa6a61e407 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:27.762605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-f10c0fdd-a025-4d57-b318-05916f9e0431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a185f5eca38344a99a5333cbac424015'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:27.797063 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-f10c0fdd-a025-4d57-b318-05916f9e0431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a185f5eca38344a99a5333cbac424015'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:27.797907 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-f10c0fdd-a025-4d57-b318-05916f9e0431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a185f5eca38344a99a5333cbac424015'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:27.798727 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-f10c0fdd-a025-4d57-b318-05916f9e0431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a185f5eca38344a99a5333cbac424015'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:27.815074 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 921/1831] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:27.817477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0c5ed-c48b-4227-a641-eefa6a61e407 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:27.825773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bab0c5ed-c48b-4227-a641-eefa6a61e407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:27.868726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 911/1832] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:27 2022] DELETE /identity/v3/users/a185f5eca38344a99a5333cbac424015 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:27.869991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c0fdd-a025-4d57-b318-05916f9e0431 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:27.874701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f10c0fdd-a025-4d57-b318-05916f9e0431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:27.897476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0c5ed-c48b-4227-a641-eefa6a61e407 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:28.003479 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:28.054043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c0fdd-a025-4d57-b318-05916f9e0431 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:28.060905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc41164-c65b-46a8-920c-eaa8d745d023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:28.062298 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc41164-c65b-46a8-920c-eaa8d745d023 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:28.062790 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc41164-c65b-46a8-920c-eaa8d745d023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:28.063087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc41164-c65b-46a8-920c-eaa8d745d023 admin admin] PATH_INFO: `/v3/projects/2535cc942657479c81cbeaa19c3170d7/users/ec6e074aae0b409f8b89e10c03a61362/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:28.070975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:28.100787 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc41164-c65b-46a8-920c-eaa8d745d023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2535cc942657479c81cbeaa19c3170d7, user_id=ec6e074aae0b409f8b89e10c03a61362, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:28.108992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc41164-c65b-46a8-920c-eaa8d745d023 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:28.124297 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d8baa-3ae3-4262-8835-3b33ce3ddbb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:28.130487 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d8baa-3ae3-4262-8835-3b33ce3ddbb1 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:28.131468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d8baa-3ae3-4262-8835-3b33ce3ddbb1 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:28.132022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d8baa-3ae3-4262-8835-3b33ce3ddbb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:28.135030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b13d8baa-3ae3-4262-8835-3b33ce3ddbb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:28.147172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 912/1833] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:27 2022] PUT /identity/v3/projects/2535cc942657479c81cbeaa19c3170d7/users/ec6e074aae0b409f8b89e10c03a61362/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:28.147172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56764bb9-4a5f-4c52-a67a-de2f2969c226 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:28.147172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56764bb9-4a5f-4c52-a67a-de2f2969c226 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:28.147172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56764bb9-4a5f-4c52-a67a-de2f2969c226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:28.164281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-56764bb9-4a5f-4c52-a67a-de2f2969c226 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:28.168225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-56764bb9-4a5f-4c52-a67a-de2f2969c226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:28.169481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d8baa-3ae3-4262-8835-3b33ce3ddbb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:28.171678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d8baa-3ae3-4262-8835-3b33ce3ddbb1 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:28.197606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 922/1834] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:28.202187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750024f6-7595-4067-a6fd-7f8be4d8cb8a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:28.204203 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750024f6-7595-4067-a6fd-7f8be4d8cb8a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:28.204708 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750024f6-7595-4067-a6fd-7f8be4d8cb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:28.278127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-750024f6-7595-4067-a6fd-7f8be4d8cb8a None None] MFA Rules not processed for user `ec6e074aae0b409f8b89e10c03a61362`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:28.281779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-750024f6-7595-4067-a6fd-7f8be4d8cb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:28.311640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 913/1835] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:28.354076 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56764bb9-4a5f-4c52-a67a-de2f2969c226 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:28.356153 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-56764bb9-4a5f-4c52-a67a-de2f2969c226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:28.558325 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56764bb9-4a5f-4c52-a67a-de2f2969c226 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:28.584845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1a46b-cb72-4ae8-9b44-a31375c08136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:28.585745 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1a46b-cb72-4ae8-9b44-a31375c08136 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:28.586045 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1a46b-cb72-4ae8-9b44-a31375c08136 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:28.586337 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1a46b-cb72-4ae8-9b44-a31375c08136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:28.603689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 923/1836] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:28.605554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1a46b-cb72-4ae8-9b44-a31375c08136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:28.608228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750024f6-7595-4067-a6fd-7f8be4d8cb8a None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:28.609023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1a46b-cb72-4ae8-9b44-a31375c08136 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:28.627023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750024f6-7595-4067-a6fd-7f8be4d8cb8a None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:28.649214 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609297d-8dbc-4753-bb13-0d801ecdd6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:28.650314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609297d-8dbc-4753-bb13-0d801ecdd6c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:28.651563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609297d-8dbc-4753-bb13-0d801ecdd6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:28.652131 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609297d-8dbc-4753-bb13-0d801ecdd6c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:28.653352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 914/1837] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:31:28.654275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609297d-8dbc-4753-bb13-0d801ecdd6c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:28.656915 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609297d-8dbc-4753-bb13-0d801ecdd6c4 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:28.729966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 924/1838] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:28 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:28.734772 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1a46b-cb72-4ae8-9b44-a31375c08136 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:28.778783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1a46b-cb72-4ae8-9b44-a31375c08136 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:29.284068 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74152f6-09e3-49e1-91ad-6193bdb50c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:29.285934 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74152f6-09e3-49e1-91ad-6193bdb50c47 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:29.286598 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74152f6-09e3-49e1-91ad-6193bdb50c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:29.287231 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74152f6-09e3-49e1-91ad-6193bdb50c47 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:29.289145 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74152f6-09e3-49e1-91ad-6193bdb50c47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:29.295343 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74152f6-09e3-49e1-91ad-6193bdb50c47 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:29.356274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 915/1839] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:28 2022] POST /identity/v3/users => generated 289 bytes in 623 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:29.357235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609297d-8dbc-4753-bb13-0d801ecdd6c4 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:29.400925 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609297d-8dbc-4753-bb13-0d801ecdd6c4 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:29.419428 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b00be-3540-49ba-a709-076e242218af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:29.425130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b00be-3540-49ba-a709-076e242218af admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:29.426092 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b00be-3540-49ba-a709-076e242218af admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:29.426893 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b00be-3540-49ba-a709-076e242218af admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:29.430982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b00be-3540-49ba-a709-076e242218af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:29.435848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b00be-3540-49ba-a709-076e242218af admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:29.466696 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 925/1840] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:29.472514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74152f6-09e3-49e1-91ad-6193bdb50c47 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:29.493130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74152f6-09e3-49e1-91ad-6193bdb50c47 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:29.500520 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aadeac0-5c17-4c21-8bed-49888eeebd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:29.501878 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadeac0-5c17-4c21-8bed-49888eeebd64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:29.502383 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadeac0-5c17-4c21-8bed-49888eeebd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:29.502707 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadeac0-5c17-4c21-8bed-49888eeebd64 admin admin] PATH_INFO: `/v3/projects/12b8cb7b8cd440e9a4dea8f1688d31c9/users/bf587de43822495da80c69551d11fd5f/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:29.535240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aadeac0-5c17-4c21-8bed-49888eeebd64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b8cb7b8cd440e9a4dea8f1688d31c9, user_id=bf587de43822495da80c69551d11fd5f, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:29.543679 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aadeac0-5c17-4c21-8bed-49888eeebd64 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:29.547021 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:29.587027 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 916/1841] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:29 2022] PUT /identity/v3/projects/12b8cb7b8cd440e9a4dea8f1688d31c9/users/bf587de43822495da80c69551d11fd5f/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:29.591285 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b00be-3540-49ba-a709-076e242218af admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:29.623075 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b00be-3540-49ba-a709-076e242218af admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:29.723533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aadeac0-5c17-4c21-8bed-49888eeebd64 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:29.727645 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0aadeac0-5c17-4c21-8bed-49888eeebd64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:29.830960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b22f82-2871-4c5b-a8ef-a17ce55c69ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:29.838151 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b22f82-2871-4c5b-a8ef-a17ce55c69ba admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:29.840689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b22f82-2871-4c5b-a8ef-a17ce55c69ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:29.841404 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b22f82-2871-4c5b-a8ef-a17ce55c69ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:29.846468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b22f82-2871-4c5b-a8ef-a17ce55c69ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:29.850452 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b22f82-2871-4c5b-a8ef-a17ce55c69ba admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:29.895864 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 926/1842] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:29.898744 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b22f82-2871-4c5b-a8ef-a17ce55c69ba admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:29.925345 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aadeac0-5c17-4c21-8bed-49888eeebd64 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:29.941646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:29.943324 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:29.943610 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:29.944170 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3 admin admin] PATH_INFO: `/v3/projects/f08aa731e21048bb9ebac0dfc7bb12e1` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:29.949630 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b22f82-2871-4c5b-a8ef-a17ce55c69ba admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:29.951456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f08aa731e21048bb9ebac0dfc7bb12e1)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:29.956172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:29.970911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6cf56-23ea-4651-833a-b724697b0f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:29.972557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6cf56-23ea-4651-833a-b724697b0f96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:29.973154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6cf56-23ea-4651-833a-b724697b0f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:29.973844 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6cf56-23ea-4651-833a-b724697b0f96 admin admin] PATH_INFO: `/v3/projects/12b8cb7b8cd440e9a4dea8f1688d31c9/users/bf587de43822495da80c69551d11fd5f/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:29.978501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6cf56-23ea-4651-833a-b724697b0f96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b8cb7b8cd440e9a4dea8f1688d31c9, user_id=bf587de43822495da80c69551d11fd5f, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:29.990652 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6cf56-23ea-4651-833a-b724697b0f96 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:30.002029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:30.009498 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 927/1843] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:29 2022] PUT /identity/v3/projects/12b8cb7b8cd440e9a4dea8f1688d31c9/users/bf587de43822495da80c69551d11fd5f/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:30.025513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ee6e9-2db9-48f9-a5a5-f416c4fd0ebd None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:30.025803 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ee6e9-2db9-48f9-a5a5-f416c4fd0ebd None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:30.029358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ee6e9-2db9-48f9-a5a5-f416c4fd0ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:30.124021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e84ee6e9-2db9-48f9-a5a5-f416c4fd0ebd None None] MFA Rules not processed for user `bf587de43822495da80c69551d11fd5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:30.129717 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e84ee6e9-2db9-48f9-a5a5-f416c4fd0ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:30.133224 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f08aa731e21048bb9ebac0dfc7bb12e1', 'request_id': 'req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:30.147429 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f08aa731e21048bb9ebac0dfc7bb12e1', 'request_id': 'req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:30.160080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f08aa731e21048bb9ebac0dfc7bb12e1', 'request_id': 'req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:30.178842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3 admin admin] The token cache is being invalidate because project f08aa731e21048bb9ebac0dfc7bb12e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:30.179120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:30.179680 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:30.184981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 917/1844] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:29 2022] DELETE /identity/v3/projects/f08aa731e21048bb9ebac0dfc7bb12e1 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:31:30.186102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:30.187888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:30.355019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 928/1845] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:30.355962 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a13cb-4897-4738-80fb-a6489ed8066f None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:30.358873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a13cb-4897-4738-80fb-a6489ed8066f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:30.359433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a13cb-4897-4738-80fb-a6489ed8066f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:30.416659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-0b0a13cb-4897-4738-80fb-a6489ed8066f None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:30.419008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0b0a13cb-4897-4738-80fb-a6489ed8066f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:30.443709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1c03a-67da-4320-8a3e-f9f2f417e7a3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:30.452430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12c742-3f12-4e7a-9fb5-552235745b80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:30.454111 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12c742-3f12-4e7a-9fb5-552235745b80 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:30.455282 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12c742-3f12-4e7a-9fb5-552235745b80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:30.456434 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12c742-3f12-4e7a-9fb5-552235745b80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:30.463365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cd12c742-3f12-4e7a-9fb5-552235745b80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:30.573616 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:30.641233 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12c742-3f12-4e7a-9fb5-552235745b80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:30.644923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12c742-3f12-4e7a-9fb5-552235745b80 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:30.660853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 929/1846] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:30.660853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a13cb-4897-4738-80fb-a6489ed8066f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:30.660853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0b0a13cb-4897-4738-80fb-a6489ed8066f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:30.665942 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 918/1847] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:30.668085 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2109b22-3a91-47c6-b364-7ab4d759b682 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:30.668631 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2109b22-3a91-47c6-b364-7ab4d759b682 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:30.669100 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2109b22-3a91-47c6-b364-7ab4d759b682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:30.692715 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a13cb-4897-4738-80fb-a6489ed8066f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:30.697722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d2109b22-3a91-47c6-b364-7ab4d759b682 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:30.699220 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d2109b22-3a91-47c6-b364-7ab4d759b682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:30.700110 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af345c8d-ca3d-4b08-a6d1-6cfb4f12eb95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:30.701197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af345c8d-ca3d-4b08-a6d1-6cfb4f12eb95 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:30.701720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af345c8d-ca3d-4b08-a6d1-6cfb4f12eb95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:30.702334 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af345c8d-ca3d-4b08-a6d1-6cfb4f12eb95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:30.714493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 919/1848] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:30.720627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2109b22-3a91-47c6-b364-7ab4d759b682 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:30.722613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af345c8d-ca3d-4b08-a6d1-6cfb4f12eb95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:30.726108 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af345c8d-ca3d-4b08-a6d1-6cfb4f12eb95 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:30.733347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2109b22-3a91-47c6-b364-7ab4d759b682 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:30.739640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f5aef-ad41-41a0-aaa6-ad85f60929e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:30.741040 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f5aef-ad41-41a0-aaa6-ad85f60929e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:30.741640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f5aef-ad41-41a0-aaa6-ad85f60929e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:30.741987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f5aef-ad41-41a0-aaa6-ad85f60929e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:30.742835 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f5aef-ad41-41a0-aaa6-ad85f60929e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:31:30.743216 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f5aef-ad41-41a0-aaa6-ad85f60929e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:30.745585 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f5aef-ad41-41a0-aaa6-ad85f60929e0 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:30.750770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 930/1849] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:30.766440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 920/1850] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:31:30.773622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af345c8d-ca3d-4b08-a6d1-6cfb4f12eb95 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:30.806720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af345c8d-ca3d-4b08-a6d1-6cfb4f12eb95 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:30.834646 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027398d-8785-4dac-9a76-8e33310a57c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:30.856149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027398d-8785-4dac-9a76-8e33310a57c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:30.857239 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027398d-8785-4dac-9a76-8e33310a57c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:30.857390 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027398d-8785-4dac-9a76-8e33310a57c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:30.858704 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027398d-8785-4dac-9a76-8e33310a57c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:30.862350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027398d-8785-4dac-9a76-8e33310a57c4 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:30.964243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f5aef-ad41-41a0-aaa6-ad85f60929e0 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:30.980433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 931/1851] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:30 2022] POST /identity/v3/projects => generated 377 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:30.984903 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027398d-8785-4dac-9a76-8e33310a57c4 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:30.986915 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f5aef-ad41-41a0-aaa6-ad85f60929e0 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:31.146102 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:31.185404 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027398d-8785-4dac-9a76-8e33310a57c4 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:31.203117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445d900-4686-4551-8bff-fe14f4674ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:31.204234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445d900-4686-4551-8bff-fe14f4674ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:31.205623 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445d900-4686-4551-8bff-fe14f4674ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:31.206059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445d900-4686-4551-8bff-fe14f4674ec6 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:31.208079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445d900-4686-4551-8bff-fe14f4674ec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:31.213882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445d900-4686-4551-8bff-fe14f4674ec6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:31.245358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bfe10-815d-416d-87b8-c84c0f9ef617 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:31.246881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bfe10-815d-416d-87b8-c84c0f9ef617 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:31.247491 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bfe10-815d-416d-87b8-c84c0f9ef617 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:31.248071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bfe10-815d-416d-87b8-c84c0f9ef617 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:31.252548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2b7bfe10-815d-416d-87b8-c84c0f9ef617 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:31.267630 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 932/1852] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:30 2022] POST /identity/v3/users => generated 289 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:31.274486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445d900-4686-4551-8bff-fe14f4674ec6 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:31.297112 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445d900-4686-4551-8bff-fe14f4674ec6 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:31.319482 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8fbe0-9e7c-4baa-9243-c319247f17aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:31.326495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8fbe0-9e7c-4baa-9243-c319247f17aa admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:31.326980 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8fbe0-9e7c-4baa-9243-c319247f17aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:31.327335 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8fbe0-9e7c-4baa-9243-c319247f17aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:31.328255 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8fbe0-9e7c-4baa-9243-c319247f17aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:31.334632 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8fbe0-9e7c-4baa-9243-c319247f17aa admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:31.389351 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 933/1853] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:31.391099 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8fbe0-9e7c-4baa-9243-c319247f17aa admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:31.440875 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8fbe0-9e7c-4baa-9243-c319247f17aa admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:31.473082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85309b23-548f-4ec9-8637-ffcfeee206f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:31.474154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85309b23-548f-4ec9-8637-ffcfeee206f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:31.475283 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85309b23-548f-4ec9-8637-ffcfeee206f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:31.475769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85309b23-548f-4ec9-8637-ffcfeee206f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:31.476758 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85309b23-548f-4ec9-8637-ffcfeee206f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:31.484832 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85309b23-548f-4ec9-8637-ffcfeee206f2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:31.545762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 934/1854] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:31.550930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85309b23-548f-4ec9-8637-ffcfeee206f2 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:31.566729 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bfe10-815d-416d-87b8-c84c0f9ef617 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:31.577230 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85309b23-548f-4ec9-8637-ffcfeee206f2 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:31.581499 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bfe10-815d-416d-87b8-c84c0f9ef617 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:31.592283 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7a038-c637-45ba-b660-9a0678e9f3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:31.593495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7a038-c637-45ba-b660-9a0678e9f3ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:31.593896 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7a038-c637-45ba-b660-9a0678e9f3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:31.594285 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7a038-c637-45ba-b660-9a0678e9f3ab admin admin] PATH_INFO: `/v3/projects/7925a9c57cdc4f9b99097fb33a00febb/users/dd5662d5a1fa4405949060b194db4ce7/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:31.642636 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7a038-c637-45ba-b660-9a0678e9f3ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=7925a9c57cdc4f9b99097fb33a00febb, user_id=dd5662d5a1fa4405949060b194db4ce7, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:31.646030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7a038-c637-45ba-b660-9a0678e9f3ab admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:31.646253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 921/1855] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:30 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:31.685330 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 935/1856] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:31 2022] PUT /identity/v3/projects/7925a9c57cdc4f9b99097fb33a00febb/users/dd5662d5a1fa4405949060b194db4ce7/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:31:31.704077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d39d2-d9c3-4ff5-a6e4-6a29715be882 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:31.704859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d39d2-d9c3-4ff5-a6e4-6a29715be882 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:31.706099 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d39d2-d9c3-4ff5-a6e4-6a29715be882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:31.848561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-bd9d39d2-d9c3-4ff5-a6e4-6a29715be882 None None] MFA Rules not processed for user `dd5662d5a1fa4405949060b194db4ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:31.851352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bd9d39d2-d9c3-4ff5-a6e4-6a29715be882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:31.952118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7a038-c637-45ba-b660-9a0678e9f3ab admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:31.954137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b8e7a038-c637-45ba-b660-9a0678e9f3ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:32.080968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7a038-c637-45ba-b660-9a0678e9f3ab admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:32.087355 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64cfc66-6e19-4e47-ab2d-483f22a0a1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:32.088485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cfc66-6e19-4e47-ab2d-483f22a0a1f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:32.088873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cfc66-6e19-4e47-ab2d-483f22a0a1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:32.089176 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cfc66-6e19-4e47-ab2d-483f22a0a1f5 admin admin] PATH_INFO: `/v3/projects/ebbae5a0810e468ebe0bc3234a635f4a` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:32.090998 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64cfc66-6e19-4e47-ab2d-483f22a0a1f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebbae5a0810e468ebe0bc3234a635f4a)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:32.093398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64cfc66-6e19-4e47-ab2d-483f22a0a1f5 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:32.124682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 922/1857] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:32.128744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d39d2-d9c3-4ff5-a6e4-6a29715be882 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:32.143610 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:32.159852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d39d2-d9c3-4ff5-a6e4-6a29715be882 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:32.171385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f8557-3651-42c1-a4d4-bac493d3255f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:32.172537 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f8557-3651-42c1-a4d4-bac493d3255f admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:32.173087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f8557-3651-42c1-a4d4-bac493d3255f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:32.173600 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f8557-3651-42c1-a4d4-bac493d3255f admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:32.174872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f8557-3651-42c1-a4d4-bac493d3255f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:32.181770 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f8557-3651-42c1-a4d4-bac493d3255f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:32.228096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c64cfc66-6e19-4e47-ab2d-483f22a0a1f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebbae5a0810e468ebe0bc3234a635f4a', 'request_id': 'req-c64cfc66-6e19-4e47-ab2d-483f22a0a1f5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:32.240971 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c64cfc66-6e19-4e47-ab2d-483f22a0a1f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebbae5a0810e468ebe0bc3234a635f4a', 'request_id': 'req-c64cfc66-6e19-4e47-ab2d-483f22a0a1f5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:32.262716 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c64cfc66-6e19-4e47-ab2d-483f22a0a1f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebbae5a0810e468ebe0bc3234a635f4a', 'request_id': 'req-c64cfc66-6e19-4e47-ab2d-483f22a0a1f5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:32.298083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c64cfc66-6e19-4e47-ab2d-483f22a0a1f5 admin admin] The token cache is being invalidate because project ebbae5a0810e468ebe0bc3234a635f4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:32.298083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c64cfc66-6e19-4e47-ab2d-483f22a0a1f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:32.299351 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c64cfc66-6e19-4e47-ab2d-483f22a0a1f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:32.305909 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 936/1858] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:31 2022] DELETE /identity/v3/projects/ebbae5a0810e468ebe0bc3234a635f4a => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:31:32.313113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11228ba5-6f17-4b47-8f26-c52d59584c80 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:32.313589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11228ba5-6f17-4b47-8f26-c52d59584c80 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:32.313888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11228ba5-6f17-4b47-8f26-c52d59584c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:32.348365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 923/1859] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:32 2022] POST /identity/v3/projects => generated 377 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:32.350931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f8557-3651-42c1-a4d4-bac493d3255f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:32.354730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-707f8557-3651-42c1-a4d4-bac493d3255f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:32.366256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-11228ba5-6f17-4b47-8f26-c52d59584c80 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:32.369371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-11228ba5-6f17-4b47-8f26-c52d59584c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:32.402627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:32.536604 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 937/1860] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:32.554911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62591b4-70b3-4cf1-80f2-452950853d87 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:32.555466 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62591b4-70b3-4cf1-80f2-452950853d87 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:32.555902 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62591b4-70b3-4cf1-80f2-452950853d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:32.594588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f8557-3651-42c1-a4d4-bac493d3255f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:32.605511 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2283e16-52c2-44cc-9735-86583063daf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:32.607004 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2283e16-52c2-44cc-9735-86583063daf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:32.607489 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2283e16-52c2-44cc-9735-86583063daf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:32.608976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2283e16-52c2-44cc-9735-86583063daf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:32.610268 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2283e16-52c2-44cc-9735-86583063daf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:32.614191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2283e16-52c2-44cc-9735-86583063daf3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:32.637941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b62591b4-70b3-4cf1-80f2-452950853d87 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:32.645260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b62591b4-70b3-4cf1-80f2-452950853d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:32.730677 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 938/1861] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:31:32.736052 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62591b4-70b3-4cf1-80f2-452950853d87 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:32.747076 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 924/1862] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:32 2022] POST /identity/v3/users => generated 289 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:32.752559 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2283e16-52c2-44cc-9735-86583063daf3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:32.815279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62591b4-70b3-4cf1-80f2-452950853d87 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:32.816180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2283e16-52c2-44cc-9735-86583063daf3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:32.850873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5fb76-3921-43f3-9098-7bc3771e16b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:32.855750 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5fb76-3921-43f3-9098-7bc3771e16b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:32.857020 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5fb76-3921-43f3-9098-7bc3771e16b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:32.859055 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5fb76-3921-43f3-9098-7bc3771e16b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:32.860226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fda243-d45c-4ed8-a9ea-070e762deebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:32.861830 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5fb76-3921-43f3-9098-7bc3771e16b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:31:32.864412 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5fb76-3921-43f3-9098-7bc3771e16b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:32.864704 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fda243-d45c-4ed8-a9ea-070e762deebd admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:32.867851 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fda243-d45c-4ed8-a9ea-070e762deebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:32.868741 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fda243-d45c-4ed8-a9ea-070e762deebd admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:32.871100 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fda243-d45c-4ed8-a9ea-070e762deebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:32.875718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5fb76-3921-43f3-9098-7bc3771e16b7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:32.880385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fda243-d45c-4ed8-a9ea-070e762deebd admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:32.939501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 939/1863] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:31:32.940450 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 925/1864] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:32.946899 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5fb76-3921-43f3-9098-7bc3771e16b7 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:32.948961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fda243-d45c-4ed8-a9ea-070e762deebd admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:32.972926 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fda243-d45c-4ed8-a9ea-070e762deebd admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:32.980212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5fb76-3921-43f3-9098-7bc3771e16b7 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:32.986184 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89535582-57de-42a3-8d7b-260c5a7d8192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:32.987533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89535582-57de-42a3-8d7b-260c5a7d8192 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:32.987863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89535582-57de-42a3-8d7b-260c5a7d8192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:32.989689 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89535582-57de-42a3-8d7b-260c5a7d8192 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:32.990683 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89535582-57de-42a3-8d7b-260c5a7d8192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:32.993106 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d018f-ee2c-406a-8ed7-8de004525b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:32.993106 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d018f-ee2c-406a-8ed7-8de004525b7c admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:32.993680 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89535582-57de-42a3-8d7b-260c5a7d8192 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:32.994048 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d018f-ee2c-406a-8ed7-8de004525b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:32.994608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d018f-ee2c-406a-8ed7-8de004525b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:32.996136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d018f-ee2c-406a-8ed7-8de004525b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:33.004101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d018f-ee2c-406a-8ed7-8de004525b7c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:33.065197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 940/1865] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:33.065779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d9b78-8ca3-4df8-80d3-7b58d8e9c5d2 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:33.066441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d9b78-8ca3-4df8-80d3-7b58d8e9c5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:33.067761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d9b78-8ca3-4df8-80d3-7b58d8e9c5d2 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:33.075245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 941/1866] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 06:31:33 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:31:33.078544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d9b78-8ca3-4df8-80d3-7b58d8e9c5d2 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:33.094285 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 926/1867] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:32 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:33.104067 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c70377-14c6-4031-81cc-dc9e9609bf7f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:33.115304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c70377-14c6-4031-81cc-dc9e9609bf7f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:33.118570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c70377-14c6-4031-81cc-dc9e9609bf7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:33.154358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d9b78-8ca3-4df8-80d3-7b58d8e9c5d2 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:33.224017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-62c70377-14c6-4031-81cc-dc9e9609bf7f None None] MFA Rules not processed for user `2a09f4878eac4d60a9a7dcedba15c05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:33.228320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-62c70377-14c6-4031-81cc-dc9e9609bf7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:33.404337 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 927/1868] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 06:31:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:33.406332 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c70377-14c6-4031-81cc-dc9e9609bf7f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:33.475861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3118455-ecdb-4e59-9b78-95939ff15f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:33.477541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3118455-ecdb-4e59-9b78-95939ff15f7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:33.478033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3118455-ecdb-4e59-9b78-95939ff15f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:33.478820 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3118455-ecdb-4e59-9b78-95939ff15f7c admin admin] PATH_INFO: `/v3/projects/0ff7e872de7141eb837ce8dc4867a2b1/users/98a85d1e9029467b9f2b0376eaa3e519/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:33.548231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3118455-ecdb-4e59-9b78-95939ff15f7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ff7e872de7141eb837ce8dc4867a2b1, user_id=98a85d1e9029467b9f2b0376eaa3e519, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:33.575432 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3118455-ecdb-4e59-9b78-95939ff15f7c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:33.590331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:33.683207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 942/1869] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:33 2022] PUT /identity/v3/projects/0ff7e872de7141eb837ce8dc4867a2b1/users/98a85d1e9029467b9f2b0376eaa3e519/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:33.693802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599056d-e09a-4592-bd38-c5f50bb1ce4d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:33.694811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599056d-e09a-4592-bd38-c5f50bb1ce4d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:33.695719 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599056d-e09a-4592-bd38-c5f50bb1ce4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:33.711163 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c70377-14c6-4031-81cc-dc9e9609bf7f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:33.823306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3599056d-e09a-4592-bd38-c5f50bb1ce4d None None] MFA Rules not processed for user `98a85d1e9029467b9f2b0376eaa3e519`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:33.829972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3599056d-e09a-4592-bd38-c5f50bb1ce4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:33.861004 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22858a-03e1-4502-b2ff-1fa51ff40b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:33.861746 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22858a-03e1-4502-b2ff-1fa51ff40b14 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:33.862046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22858a-03e1-4502-b2ff-1fa51ff40b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:33.862332 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22858a-03e1-4502-b2ff-1fa51ff40b14 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:33.863374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22858a-03e1-4502-b2ff-1fa51ff40b14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:33.865928 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22858a-03e1-4502-b2ff-1fa51ff40b14 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:33.909419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 928/1870] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:33 2022] POST /identity/v3/users => generated 288 bytes in 501 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:33.910791 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22858a-03e1-4502-b2ff-1fa51ff40b14 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:33.938320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22858a-03e1-4502-b2ff-1fa51ff40b14 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:33.947524 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe344999-1e67-4ab5-b958-dbddbafbd69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:33.948247 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe344999-1e67-4ab5-b958-dbddbafbd69a admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:33.948540 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe344999-1e67-4ab5-b958-dbddbafbd69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:33.948828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe344999-1e67-4ab5-b958-dbddbafbd69a admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:33.949613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe344999-1e67-4ab5-b958-dbddbafbd69a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:33.951302 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe344999-1e67-4ab5-b958-dbddbafbd69a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:33.987640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 929/1871] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:33.988133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe344999-1e67-4ab5-b958-dbddbafbd69a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:34.012505 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe344999-1e67-4ab5-b958-dbddbafbd69a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:34.028382 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb892c5-8614-41e1-ae99-eabdf83a7d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:34.030088 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb892c5-8614-41e1-ae99-eabdf83a7d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:34.030088 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb892c5-8614-41e1-ae99-eabdf83a7d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:34.030088 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb892c5-8614-41e1-ae99-eabdf83a7d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:34.031014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb892c5-8614-41e1-ae99-eabdf83a7d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:34.033120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb892c5-8614-41e1-ae99-eabdf83a7d72 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:34.066881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 943/1872] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:34.067419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 930/1873] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:31:34.070728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599056d-e09a-4592-bd38-c5f50bb1ce4d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:34.070883 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb892c5-8614-41e1-ae99-eabdf83a7d72 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:34.085746 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb892c5-8614-41e1-ae99-eabdf83a7d72 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:34.091547 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301c48c-8cd4-47fb-8cbe-688781524c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:34.093060 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301c48c-8cd4-47fb-8cbe-688781524c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:34.094008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301c48c-8cd4-47fb-8cbe-688781524c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:34.094299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301c48c-8cd4-47fb-8cbe-688781524c41 admin admin] PATH_INFO: `/v3/projects/e636eba5969b4bcfbccff936b37d5049/users/8dadf9fc1d974ced9bc4a76282f699b3/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:34.105145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599056d-e09a-4592-bd38-c5f50bb1ce4d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:34.120840 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301c48c-8cd4-47fb-8cbe-688781524c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e636eba5969b4bcfbccff936b37d5049, user_id=8dadf9fc1d974ced9bc4a76282f699b3, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:34.127764 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301c48c-8cd4-47fb-8cbe-688781524c41 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:34.150710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff596f-c21e-4688-b034-d8e9dea70dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:34.152816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff596f-c21e-4688-b034-d8e9dea70dcd admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:34.153511 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff596f-c21e-4688-b034-d8e9dea70dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:34.155108 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff596f-c21e-4688-b034-d8e9dea70dcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:34.160705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff596f-c21e-4688-b034-d8e9dea70dcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:34.166347 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff596f-c21e-4688-b034-d8e9dea70dcd admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:34.175287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:34.188550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 931/1874] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:34 2022] PUT /identity/v3/projects/e636eba5969b4bcfbccff936b37d5049/users/8dadf9fc1d974ced9bc4a76282f699b3/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:31:34.195593 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301c48c-8cd4-47fb-8cbe-688781524c41 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:34.204012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d301c48c-8cd4-47fb-8cbe-688781524c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:34.249466 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 944/1875] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:34 2022] POST /identity/v3/projects => generated 377 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:34.251198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086efca8-80dc-4975-95b7-78afddd62fa1 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:34.251502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086efca8-80dc-4975-95b7-78afddd62fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:34.251785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086efca8-80dc-4975-95b7-78afddd62fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:34.342331 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-086efca8-80dc-4975-95b7-78afddd62fa1 None None] MFA Rules not processed for user `8dadf9fc1d974ced9bc4a76282f699b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:34.345383 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-086efca8-80dc-4975-95b7-78afddd62fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:34.423178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301c48c-8cd4-47fb-8cbe-688781524c41 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:34.472779 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:34.562143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 945/1876] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:34.562143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086efca8-80dc-4975-95b7-78afddd62fa1 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:34.596803 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05099d7f-46e3-4310-a8b0-fae34e9d3989 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:34.597526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05099d7f-46e3-4310-a8b0-fae34e9d3989 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:34.597843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05099d7f-46e3-4310-a8b0-fae34e9d3989 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:34.598091 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05099d7f-46e3-4310-a8b0-fae34e9d3989 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:34.629775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05099d7f-46e3-4310-a8b0-fae34e9d3989 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:34.632776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05099d7f-46e3-4310-a8b0-fae34e9d3989 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:34.664142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 932/1877] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:34.810987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05099d7f-46e3-4310-a8b0-fae34e9d3989 service nova] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:34.813133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-05099d7f-46e3-4310-a8b0-fae34e9d3989 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:34.829044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086efca8-80dc-4975-95b7-78afddd62fa1 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:34.851133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a671ff-3542-4a05-9058-e5a35fb75938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:34.852860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a671ff-3542-4a05-9058-e5a35fb75938 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:34.852860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a671ff-3542-4a05-9058-e5a35fb75938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:34.853219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a671ff-3542-4a05-9058-e5a35fb75938 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:34.859599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a671ff-3542-4a05-9058-e5a35fb75938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:34.865713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a671ff-3542-4a05-9058-e5a35fb75938 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:34.921759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 946/1878] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:34 2022] POST /identity/v3/users => generated 295 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:34.925616 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a671ff-3542-4a05-9058-e5a35fb75938 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:34.947121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a671ff-3542-4a05-9058-e5a35fb75938 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:34.963777 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c314e3-a111-4a09-8389-dbd5eeac7e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:34.964630 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c314e3-a111-4a09-8389-dbd5eeac7e36 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:34.964925 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c314e3-a111-4a09-8389-dbd5eeac7e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:34.965309 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c314e3-a111-4a09-8389-dbd5eeac7e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:34.966014 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c314e3-a111-4a09-8389-dbd5eeac7e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:34.967697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c314e3-a111-4a09-8389-dbd5eeac7e36 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:34.992872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05099d7f-46e3-4310-a8b0-fae34e9d3989 service nova] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:35.009441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 947/1879] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:35.009531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74bd1a8-ae84-4437-bcb8-baec823eb9c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:35.009531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bd1a8-ae84-4437-bcb8-baec823eb9c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:35.009531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bd1a8-ae84-4437-bcb8-baec823eb9c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:35.009531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bd1a8-ae84-4437-bcb8-baec823eb9c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:35.012133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c314e3-a111-4a09-8389-dbd5eeac7e36 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:35.052066 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74bd1a8-ae84-4437-bcb8-baec823eb9c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:35.060302 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74bd1a8-ae84-4437-bcb8-baec823eb9c4 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:35.095863 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c314e3-a111-4a09-8389-dbd5eeac7e36 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:35.102588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 933/1880] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:35.154919 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee1f24-98b6-4e22-84b3-c4558bd19140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:35.165569 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee1f24-98b6-4e22-84b3-c4558bd19140 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:35.167154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee1f24-98b6-4e22-84b3-c4558bd19140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:35.167682 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee1f24-98b6-4e22-84b3-c4558bd19140 admin admin] PATH_INFO: `/v3/projects/09b2030a8f4d45d3ac37ff1640c14dcb/users/84e453c62d924756adec5bba8994cb80/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:35.291336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee1f24-98b6-4e22-84b3-c4558bd19140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b2030a8f4d45d3ac37ff1640c14dcb, user_id=84e453c62d924756adec5bba8994cb80, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:35.296895 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee1f24-98b6-4e22-84b3-c4558bd19140 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:35.385728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 948/1881] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:35 2022] PUT /identity/v3/projects/09b2030a8f4d45d3ac37ff1640c14dcb/users/84e453c62d924756adec5bba8994cb80/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:35.389740 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74bd1a8-ae84-4437-bcb8-baec823eb9c4 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:35.418915 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74bd1a8-ae84-4437-bcb8-baec823eb9c4 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:35.638572 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fd0b3-cd69-4fd5-97b8-4e21081134f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:35.644586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fd0b3-cd69-4fd5-97b8-4e21081134f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:35.645240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fd0b3-cd69-4fd5-97b8-4e21081134f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:35.645860 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fd0b3-cd69-4fd5-97b8-4e21081134f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:35.651642 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315fd0b3-cd69-4fd5-97b8-4e21081134f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:35.661777 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315fd0b3-cd69-4fd5-97b8-4e21081134f2 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:35.713404 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 934/1882] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:31:35.713850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee1f24-98b6-4e22-84b3-c4558bd19140 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:35.742286 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee1f24-98b6-4e22-84b3-c4558bd19140 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:35.780936 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc27317-e9d4-4633-953f-925b91bb869f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:35.782802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc27317-e9d4-4633-953f-925b91bb869f admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:35.783014 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc27317-e9d4-4633-953f-925b91bb869f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:35.788256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc27317-e9d4-4633-953f-925b91bb869f admin admin] PATH_INFO: `/v3/projects/09b2030a8f4d45d3ac37ff1640c14dcb/users/84e453c62d924756adec5bba8994cb80/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:35.796340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc27317-e9d4-4633-953f-925b91bb869f admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b2030a8f4d45d3ac37ff1640c14dcb, user_id=84e453c62d924756adec5bba8994cb80, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:35.813148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc27317-e9d4-4633-953f-925b91bb869f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:35.870510 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 949/1883] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:35 2022] PUT /identity/v3/projects/09b2030a8f4d45d3ac37ff1640c14dcb/users/84e453c62d924756adec5bba8994cb80/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:35.897756 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076321b0-a2ef-4536-a691-4ca3825b8789 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:35.897756 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076321b0-a2ef-4536-a691-4ca3825b8789 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:35.897756 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076321b0-a2ef-4536-a691-4ca3825b8789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:36.048695 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-076321b0-a2ef-4536-a691-4ca3825b8789 None None] MFA Rules not processed for user `84e453c62d924756adec5bba8994cb80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:36.051208 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-076321b0-a2ef-4536-a691-4ca3825b8789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:36.093114 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc27317-e9d4-4633-953f-925b91bb869f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:36.098672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cfc27317-e9d4-4633-953f-925b91bb869f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:36.234610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:36.306722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:36.536627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc27317-e9d4-4633-953f-925b91bb869f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:36.538000 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 935/1884] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:35 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 644 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:36.551551 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c10268-871f-4df2-9455-1ee9e822932c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:36.558132 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c10268-871f-4df2-9455-1ee9e822932c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:36.560154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c10268-871f-4df2-9455-1ee9e822932c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:36.560618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c10268-871f-4df2-9455-1ee9e822932c admin admin] PATH_INFO: `/v3/users/32c3249883a4401abe8c1941febe7cb2` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:36.566550 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c10268-871f-4df2-9455-1ee9e822932c admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c3249883a4401abe8c1941febe7cb2)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:36.577558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c10268-871f-4df2-9455-1ee9e822932c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:36.593555 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076321b0-a2ef-4536-a691-4ca3825b8789 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:36.596276 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-076321b0-a2ef-4536-a691-4ca3825b8789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:36.759187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076321b0-a2ef-4536-a691-4ca3825b8789 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:36.766941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-97c10268-871f-4df2-9455-1ee9e822932c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c3249883a4401abe8c1941febe7cb2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:36.770287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e9ae1-4839-41a0-89fc-0e03aac2aeaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:36.770964 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e9ae1-4839-41a0-89fc-0e03aac2aeaa service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:36.772522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e9ae1-4839-41a0-89fc-0e03aac2aeaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:36.772822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e9ae1-4839-41a0-89fc-0e03aac2aeaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:36.774000 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-97c10268-871f-4df2-9455-1ee9e822932c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c3249883a4401abe8c1941febe7cb2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:36.777988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9f7e9ae1-4839-41a0-89fc-0e03aac2aeaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:36.808198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e9ae1-4839-41a0-89fc-0e03aac2aeaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:36.809781 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e9ae1-4839-41a0-89fc-0e03aac2aeaa service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:36.827146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 936/1885] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:31:36.852991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-97c10268-871f-4df2-9455-1ee9e822932c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c3249883a4401abe8c1941febe7cb2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:36.855025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-97c10268-871f-4df2-9455-1ee9e822932c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c3249883a4401abe8c1941febe7cb2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:36.874642 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-97c10268-871f-4df2-9455-1ee9e822932c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c3249883a4401abe8c1941febe7cb2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:36.941797 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 950/1886] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:36 2022] DELETE /identity/v3/users/32c3249883a4401abe8c1941febe7cb2 => generated 0 bytes in 844 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:36.942039 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1bbcb-d1bf-4281-9250-ceff81cffa20 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:36.942338 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1bbcb-d1bf-4281-9250-ceff81cffa20 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:36.943821 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1bbcb-d1bf-4281-9250-ceff81cffa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:36.987745 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-ccb1bbcb-d1bf-4281-9250-ceff81cffa20 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:36.989390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ccb1bbcb-d1bf-4281-9250-ceff81cffa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:37.039838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c10268-871f-4df2-9455-1ee9e822932c admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:37.044137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-97c10268-871f-4df2-9455-1ee9e822932c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:37.200681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 937/1887] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:31:37.247575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1bbcb-d1bf-4281-9250-ceff81cffa20 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:37.249461 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ccb1bbcb-d1bf-4281-9250-ceff81cffa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:37.277689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c10268-871f-4df2-9455-1ee9e822932c admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:37.294361 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd09fe-dab1-4ac8-88a1-40a4282f8141 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:37.295105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd09fe-dab1-4ac8-88a1-40a4282f8141 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:37.296000 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd09fe-dab1-4ac8-88a1-40a4282f8141 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:37.297146 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd09fe-dab1-4ac8-88a1-40a4282f8141 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:37.320208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b4fd09fe-dab1-4ac8-88a1-40a4282f8141 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:37.392799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1bbcb-d1bf-4281-9250-ceff81cffa20 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:37.401245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5abb4f-4791-4bde-a60a-3468fa87dc7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:37.402132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5abb4f-4791-4bde-a60a-3468fa87dc7b service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:37.402411 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5abb4f-4791-4bde-a60a-3468fa87dc7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:37.402854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5abb4f-4791-4bde-a60a-3468fa87dc7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:37.425632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5abb4f-4791-4bde-a60a-3468fa87dc7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:37.429340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5abb4f-4791-4bde-a60a-3468fa87dc7b service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:37.446214 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:37.471132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 938/1888] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:31:37.486205 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd09fe-dab1-4ac8-88a1-40a4282f8141 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:37.487613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd09fe-dab1-4ac8-88a1-40a4282f8141 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:37.538407 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 951/1889] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:31:38.209317 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5abb4f-4791-4bde-a60a-3468fa87dc7b service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:38.211797 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8c5abb4f-4791-4bde-a60a-3468fa87dc7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:38.263475 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5abb4f-4791-4bde-a60a-3468fa87dc7b service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:38.272853 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a88050-3e70-4cce-83c6-adb74e1d3b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:38.277107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a88050-3e70-4cce-83c6-adb74e1d3b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:38.277637 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a88050-3e70-4cce-83c6-adb74e1d3b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:38.293624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a88050-3e70-4cce-83c6-adb74e1d3b2a admin admin] PATH_INFO: `/v3/projects/f25221589bf04cdaaaa799af6f2bfd77` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:38.293624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a88050-3e70-4cce-83c6-adb74e1d3b2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f25221589bf04cdaaaa799af6f2bfd77)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:38.296588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a88050-3e70-4cce-83c6-adb74e1d3b2a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:38.317015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:38.421777 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-18a88050-3e70-4cce-83c6-adb74e1d3b2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f25221589bf04cdaaaa799af6f2bfd77', 'request_id': 'req-18a88050-3e70-4cce-83c6-adb74e1d3b2a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:38.441098 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-18a88050-3e70-4cce-83c6-adb74e1d3b2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f25221589bf04cdaaaa799af6f2bfd77', 'request_id': 'req-18a88050-3e70-4cce-83c6-adb74e1d3b2a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:38.454218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-18a88050-3e70-4cce-83c6-adb74e1d3b2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f25221589bf04cdaaaa799af6f2bfd77', 'request_id': 'req-18a88050-3e70-4cce-83c6-adb74e1d3b2a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:38.489040 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-18a88050-3e70-4cce-83c6-adb74e1d3b2a admin admin] The token cache is being invalidate because project f25221589bf04cdaaaa799af6f2bfd77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:38.489562 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-18a88050-3e70-4cce-83c6-adb74e1d3b2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:38.490865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-18a88050-3e70-4cce-83c6-adb74e1d3b2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:38.497398 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 939/1890] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:38 2022] DELETE /identity/v3/projects/f25221589bf04cdaaaa799af6f2bfd77 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:38.504472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd09fe-dab1-4ac8-88a1-40a4282f8141 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:38.528432 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd09fe-dab1-4ac8-88a1-40a4282f8141 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:38.673394 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c60cf-a1d5-4a6b-b20c-2c4003df66a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:38.674882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c60cf-a1d5-4a6b-b20c-2c4003df66a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:38.675312 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c60cf-a1d5-4a6b-b20c-2c4003df66a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:38.675775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c60cf-a1d5-4a6b-b20c-2c4003df66a7 admin admin] PATH_INFO: `/v3/users/7cced47d0020445b9ce5eae9aa908d55` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:38.689336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c60cf-a1d5-4a6b-b20c-2c4003df66a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cced47d0020445b9ce5eae9aa908d55)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:38.693789 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c60cf-a1d5-4a6b-b20c-2c4003df66a7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:38.798627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-046c60cf-a1d5-4a6b-b20c-2c4003df66a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cced47d0020445b9ce5eae9aa908d55'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:38.801488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-046c60cf-a1d5-4a6b-b20c-2c4003df66a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cced47d0020445b9ce5eae9aa908d55'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:38.823842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-046c60cf-a1d5-4a6b-b20c-2c4003df66a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cced47d0020445b9ce5eae9aa908d55'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:38.824147 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-046c60cf-a1d5-4a6b-b20c-2c4003df66a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cced47d0020445b9ce5eae9aa908d55'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:38.845841 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-046c60cf-a1d5-4a6b-b20c-2c4003df66a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cced47d0020445b9ce5eae9aa908d55'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:38.875027 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 952/1891] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:38 2022] DELETE /identity/v3/users/7cced47d0020445b9ce5eae9aa908d55 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:39.376716 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a88050-3e70-4cce-83c6-adb74e1d3b2a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:39.378878 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-18a88050-3e70-4cce-83c6-adb74e1d3b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:39.397794 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a88050-3e70-4cce-83c6-adb74e1d3b2a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:39.485836 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:39.486978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:39.487374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:39.487797 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec admin admin] PATH_INFO: `/v3/projects/2a97e366c04345ecb93236454316b2fb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:39.489433 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a97e366c04345ecb93236454316b2fb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:39.492816 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:39.549295 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a97e366c04345ecb93236454316b2fb', 'request_id': 'req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:39.559601 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a97e366c04345ecb93236454316b2fb', 'request_id': 'req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:39.567468 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a97e366c04345ecb93236454316b2fb', 'request_id': 'req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:39.580977 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec admin admin] The token cache is being invalidate because project 2a97e366c04345ecb93236454316b2fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:39.581271 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:39.581974 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:39.583695 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 940/1892] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:39 2022] DELETE /identity/v3/projects/2a97e366c04345ecb93236454316b2fb => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:39.588387 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c60cf-a1d5-4a6b-b20c-2c4003df66a7 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:39.590063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-046c60cf-a1d5-4a6b-b20c-2c4003df66a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:39.604079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c60cf-a1d5-4a6b-b20c-2c4003df66a7 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:39.667221 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115af67-873e-42b1-b8ac-314a807bb3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:39.668029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115af67-873e-42b1-b8ac-314a807bb3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:39.668315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115af67-873e-42b1-b8ac-314a807bb3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:39.668593 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115af67-873e-42b1-b8ac-314a807bb3f8 admin admin] PATH_INFO: `/v3/users/b16db87abb044ab294071e5a9701cb86` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:39.669940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3115af67-873e-42b1-b8ac-314a807bb3f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b16db87abb044ab294071e5a9701cb86)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:39.672122 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3115af67-873e-42b1-b8ac-314a807bb3f8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:39.685070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:39.729960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3115af67-873e-42b1-b8ac-314a807bb3f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b16db87abb044ab294071e5a9701cb86'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:39.730634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3115af67-873e-42b1-b8ac-314a807bb3f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b16db87abb044ab294071e5a9701cb86'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:39.744012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3115af67-873e-42b1-b8ac-314a807bb3f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b16db87abb044ab294071e5a9701cb86'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:39.744755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3115af67-873e-42b1-b8ac-314a807bb3f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b16db87abb044ab294071e5a9701cb86'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:39.754364 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3115af67-873e-42b1-b8ac-314a807bb3f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b16db87abb044ab294071e5a9701cb86'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:39.771131 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 953/1893] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:39 2022] DELETE /identity/v3/users/b16db87abb044ab294071e5a9701cb86 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:40.274363 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:40.278164 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:40.356288 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ad496-ab07-41a0-a225-84bcfd1c6eec admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:40.428531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:40.518209 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afe599-828a-4a0e-afba-73fee72e5cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:40.519407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afe599-828a-4a0e-afba-73fee72e5cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:40.520745 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afe599-828a-4a0e-afba-73fee72e5cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:40.521500 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afe599-828a-4a0e-afba-73fee72e5cab admin admin] PATH_INFO: `/v3/projects/9fca6ca1761540e8a265abdf8373bc15` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:40.525355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6afe599-828a-4a0e-afba-73fee72e5cab admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fca6ca1761540e8a265abdf8373bc15)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:40.532535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6afe599-828a-4a0e-afba-73fee72e5cab admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:40.711216 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a6afe599-828a-4a0e-afba-73fee72e5cab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fca6ca1761540e8a265abdf8373bc15', 'request_id': 'req-a6afe599-828a-4a0e-afba-73fee72e5cab'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:40.726395 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a6afe599-828a-4a0e-afba-73fee72e5cab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fca6ca1761540e8a265abdf8373bc15', 'request_id': 'req-a6afe599-828a-4a0e-afba-73fee72e5cab'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:40.741775 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a6afe599-828a-4a0e-afba-73fee72e5cab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fca6ca1761540e8a265abdf8373bc15', 'request_id': 'req-a6afe599-828a-4a0e-afba-73fee72e5cab'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:40.774890 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a6afe599-828a-4a0e-afba-73fee72e5cab admin admin] The token cache is being invalidate because project 9fca6ca1761540e8a265abdf8373bc15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:40.775221 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a6afe599-828a-4a0e-afba-73fee72e5cab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:40.776374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a6afe599-828a-4a0e-afba-73fee72e5cab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:40.778620 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 941/1894] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:40 2022] DELETE /identity/v3/projects/9fca6ca1761540e8a265abdf8373bc15 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:40.785858 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89c4ba-1a05-4a7d-bdbd-e1200a5f82f7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:40.787223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89c4ba-1a05-4a7d-bdbd-e1200a5f82f7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:40.787854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89c4ba-1a05-4a7d-bdbd-e1200a5f82f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:40.829997 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-bf89c4ba-1a05-4a7d-bdbd-e1200a5f82f7 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:40.832202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bf89c4ba-1a05-4a7d-bdbd-e1200a5f82f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:40.986841 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 954/1895] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:40.998779 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d41859-66b5-4921-afa8-728c3fb2858c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:40.998779 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d41859-66b5-4921-afa8-728c3fb2858c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:40.998779 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d41859-66b5-4921-afa8-728c3fb2858c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:41.030109 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-20d41859-66b5-4921-afa8-728c3fb2858c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:41.032815 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-20d41859-66b5-4921-afa8-728c3fb2858c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:41.045494 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 942/1896] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:41.050161 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89c4ba-1a05-4a7d-bdbd-e1200a5f82f7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:41.074523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89c4ba-1a05-4a7d-bdbd-e1200a5f82f7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:41.088088 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb34438-67be-4222-8ec2-f2bfb7069ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:41.089118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb34438-67be-4222-8ec2-f2bfb7069ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:41.090512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb34438-67be-4222-8ec2-f2bfb7069ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:41.090842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb34438-67be-4222-8ec2-f2bfb7069ed4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:41.093377 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb34438-67be-4222-8ec2-f2bfb7069ed4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:31:41.093809 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb34438-67be-4222-8ec2-f2bfb7069ed4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:41.098444 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb34438-67be-4222-8ec2-f2bfb7069ed4 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:41.135459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d41859-66b5-4921-afa8-728c3fb2858c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:41.139914 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 955/1897] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:31:41.159208 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d41859-66b5-4921-afa8-728c3fb2858c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:41.173866 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a7b26-d8a1-48a6-996f-566ded56f8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:41.174788 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a7b26-d8a1-48a6-996f-566ded56f8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:41.175098 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a7b26-d8a1-48a6-996f-566ded56f8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:41.175450 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a7b26-d8a1-48a6-996f-566ded56f8f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:41.176285 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a7b26-d8a1-48a6-996f-566ded56f8f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:41.180517 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a7b26-d8a1-48a6-996f-566ded56f8f9 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:41.228046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 943/1898] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:41 2022] POST /identity/v3/projects => generated 391 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:41.232447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb34438-67be-4222-8ec2-f2bfb7069ed4 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:41.274766 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb34438-67be-4222-8ec2-f2bfb7069ed4 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:41.304499 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a7b26-d8a1-48a6-996f-566ded56f8f9 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:41.307082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0d3a7b26-d8a1-48a6-996f-566ded56f8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:41.456379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51489aed-aa02-4ec4-8a84-4ba0f8b4343b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:41.457592 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51489aed-aa02-4ec4-8a84-4ba0f8b4343b admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:41.458146 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51489aed-aa02-4ec4-8a84-4ba0f8b4343b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:41.459797 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51489aed-aa02-4ec4-8a84-4ba0f8b4343b admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:41.460869 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51489aed-aa02-4ec4-8a84-4ba0f8b4343b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:41.463221 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51489aed-aa02-4ec4-8a84-4ba0f8b4343b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:41.501795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 956/1899] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:41 2022] POST /identity/v3/users => generated 296 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:41.506183 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51489aed-aa02-4ec4-8a84-4ba0f8b4343b admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:41.526545 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:41.537896 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51489aed-aa02-4ec4-8a84-4ba0f8b4343b admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:41.555536 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20660d0b-a36f-4999-b3e6-1678d90e1d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:41.557355 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20660d0b-a36f-4999-b3e6-1678d90e1d5e admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:41.558363 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20660d0b-a36f-4999-b3e6-1678d90e1d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:41.558662 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20660d0b-a36f-4999-b3e6-1678d90e1d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:41.559949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20660d0b-a36f-4999-b3e6-1678d90e1d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:41.561762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20660d0b-a36f-4999-b3e6-1678d90e1d5e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:41.596821 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 957/1900] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:41.597934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20660d0b-a36f-4999-b3e6-1678d90e1d5e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:41.627220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20660d0b-a36f-4999-b3e6-1678d90e1d5e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:41.651342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef81e0-5f56-4bfc-85ee-5c2286905bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:41.652710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef81e0-5f56-4bfc-85ee-5c2286905bac admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:41.653237 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef81e0-5f56-4bfc-85ee-5c2286905bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:41.653817 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef81e0-5f56-4bfc-85ee-5c2286905bac admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:41.655703 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef81e0-5f56-4bfc-85ee-5c2286905bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:41.659098 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef81e0-5f56-4bfc-85ee-5c2286905bac admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:41.668818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a7b26-d8a1-48a6-996f-566ded56f8f9 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:41.676879 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df6b55-4e4c-44a0-ac92-9ffc51894127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:41.678907 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df6b55-4e4c-44a0-ac92-9ffc51894127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:41.679566 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df6b55-4e4c-44a0-ac92-9ffc51894127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:41.680535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df6b55-4e4c-44a0-ac92-9ffc51894127 admin admin] PATH_INFO: `/v3/users/8dadf9fc1d974ced9bc4a76282f699b3` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:41.682863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df6b55-4e4c-44a0-ac92-9ffc51894127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dadf9fc1d974ced9bc4a76282f699b3)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:41.686293 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df6b55-4e4c-44a0-ac92-9ffc51894127 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:41.713433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 958/1901] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:41.719941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef81e0-5f56-4bfc-85ee-5c2286905bac admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:41.752551 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef81e0-5f56-4bfc-85ee-5c2286905bac admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:41.769065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cd30a-3e6d-4715-bc6b-95a76c9e85a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:41.772748 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cd30a-3e6d-4715-bc6b-95a76c9e85a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:41.773467 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cd30a-3e6d-4715-bc6b-95a76c9e85a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:41.773967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cd30a-3e6d-4715-bc6b-95a76c9e85a0 admin admin] PATH_INFO: `/v3/projects/d9166848cb5e4137bcfe3c5acc26882e/users/e34a7e359dff4673b00dced96373fbbb/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:41.785837 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:41.814009 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0cd30a-3e6d-4715-bc6b-95a76c9e85a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9166848cb5e4137bcfe3c5acc26882e, user_id=e34a7e359dff4673b00dced96373fbbb, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:41.818445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0cd30a-3e6d-4715-bc6b-95a76c9e85a0 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:41.830904 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-88df6b55-4e4c-44a0-ac92-9ffc51894127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dadf9fc1d974ced9bc4a76282f699b3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:41.847947 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 959/1902] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:41 2022] PUT /identity/v3/projects/d9166848cb5e4137bcfe3c5acc26882e/users/e34a7e359dff4673b00dced96373fbbb/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:31:41.851935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-88df6b55-4e4c-44a0-ac92-9ffc51894127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dadf9fc1d974ced9bc4a76282f699b3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:41.867319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-88df6b55-4e4c-44a0-ac92-9ffc51894127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dadf9fc1d974ced9bc4a76282f699b3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:41.871256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-88df6b55-4e4c-44a0-ac92-9ffc51894127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dadf9fc1d974ced9bc4a76282f699b3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:41.873883 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-88df6b55-4e4c-44a0-ac92-9ffc51894127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dadf9fc1d974ced9bc4a76282f699b3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:41.874796 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1cd32-b2e6-4c53-8d59-d087b5df3dfd None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:41.875116 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1cd32-b2e6-4c53-8d59-d087b5df3dfd None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:41.875432 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1cd32-b2e6-4c53-8d59-d087b5df3dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:41.908895 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 944/1903] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:41 2022] DELETE /identity/v3/users/8dadf9fc1d974ced9bc4a76282f699b3 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:41.911399 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3180521-eced-49aa-8b15-d7ee843035c7 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:41.912046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3180521-eced-49aa-8b15-d7ee843035c7 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:41.913257 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3180521-eced-49aa-8b15-d7ee843035c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:41.938139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-0ac1cd32-b2e6-4c53-8d59-d087b5df3dfd None None] MFA Rules not processed for user `e34a7e359dff4673b00dced96373fbbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:41.940133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0ac1cd32-b2e6-4c53-8d59-d087b5df3dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:41.955154 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f3180521-eced-49aa-8b15-d7ee843035c7 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:41.956652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f3180521-eced-49aa-8b15-d7ee843035c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:42.093602 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 945/1904] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:42.097400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3180521-eced-49aa-8b15-d7ee843035c7 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:42.107777 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f3180521-eced-49aa-8b15-d7ee843035c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:42.132130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 960/1905] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:42.133985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1cd32-b2e6-4c53-8d59-d087b5df3dfd None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:42.136664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0ac1cd32-b2e6-4c53-8d59-d087b5df3dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:42.150221 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3180521-eced-49aa-8b15-d7ee843035c7 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:42.163599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac9596-b7b7-4809-b36d-4458e88647a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:42.163599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac9596-b7b7-4809-b36d-4458e88647a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:42.163599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac9596-b7b7-4809-b36d-4458e88647a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:42.163599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac9596-b7b7-4809-b36d-4458e88647a2 admin admin] PATH_INFO: `/v3/users/dd5662d5a1fa4405949060b194db4ce7` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:42.163879 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac9596-b7b7-4809-b36d-4458e88647a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd5662d5a1fa4405949060b194db4ce7)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:42.165466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac9596-b7b7-4809-b36d-4458e88647a2 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:42.254430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0dac9596-b7b7-4809-b36d-4458e88647a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd5662d5a1fa4405949060b194db4ce7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:42.274973 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1cd32-b2e6-4c53-8d59-d087b5df3dfd None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:42.278980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0dac9596-b7b7-4809-b36d-4458e88647a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd5662d5a1fa4405949060b194db4ce7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:42.283951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed013fc-a60f-42fc-a7ae-e39b0b8be45d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:42.285651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed013fc-a60f-42fc-a7ae-e39b0b8be45d service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:42.286394 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed013fc-a60f-42fc-a7ae-e39b0b8be45d service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:42.287549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed013fc-a60f-42fc-a7ae-e39b0b8be45d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:42.297086 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0dac9596-b7b7-4809-b36d-4458e88647a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd5662d5a1fa4405949060b194db4ce7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:42.298723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0dac9596-b7b7-4809-b36d-4458e88647a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd5662d5a1fa4405949060b194db4ce7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:42.299707 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-0dac9596-b7b7-4809-b36d-4458e88647a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd5662d5a1fa4405949060b194db4ce7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:42.308481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed013fc-a60f-42fc-a7ae-e39b0b8be45d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:42.311828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed013fc-a60f-42fc-a7ae-e39b0b8be45d service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:42.330641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 946/1906] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:42 2022] DELETE /identity/v3/users/dd5662d5a1fa4405949060b194db4ce7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:42.332059 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac9596-b7b7-4809-b36d-4458e88647a2 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:42.342099 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0dac9596-b7b7-4809-b36d-4458e88647a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:42.343924 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 961/1907] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:42.348249 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3fcdf-c373-4e59-9355-7249673c134c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:42.348941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3fcdf-c373-4e59-9355-7249673c134c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:42.350291 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3fcdf-c373-4e59-9355-7249673c134c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:42.380355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:42.397198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b0b3fcdf-c373-4e59-9355-7249673c134c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:42.399827 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b0b3fcdf-c373-4e59-9355-7249673c134c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:42.467395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:42.482092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac9596-b7b7-4809-b36d-4458e88647a2 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:42.488789 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 962/1908] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:42.497819 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38a3e7-04db-4769-ab61-786805cc885d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:42.499006 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38a3e7-04db-4769-ab61-786805cc885d admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:42.499524 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38a3e7-04db-4769-ab61-786805cc885d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:42.500268 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38a3e7-04db-4769-ab61-786805cc885d admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:42.501663 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af38a3e7-04db-4769-ab61-786805cc885d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:42.506068 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af38a3e7-04db-4769-ab61-786805cc885d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:42.513719 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3fcdf-c373-4e59-9355-7249673c134c None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:42.516104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b0b3fcdf-c373-4e59-9355-7249673c134c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:42.570353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 947/1909] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:42 2022] POST /identity/v3/projects => generated 393 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:42.572094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38a3e7-04db-4769-ab61-786805cc885d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:42.612260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3fcdf-c373-4e59-9355-7249673c134c None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:42.701189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38a3e7-04db-4769-ab61-786805cc885d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:42.707020 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a952482-9714-4988-9d94-6ecc0a0bd569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:42.707865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a952482-9714-4988-9d94-6ecc0a0bd569 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:42.708954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a952482-9714-4988-9d94-6ecc0a0bd569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:42.709570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a952482-9714-4988-9d94-6ecc0a0bd569 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:42.710591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a952482-9714-4988-9d94-6ecc0a0bd569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:42.712728 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a952482-9714-4988-9d94-6ecc0a0bd569 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:42.727747 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ef176-33c5-4b31-93a0-bc37a50566c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:42.728592 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ef176-33c5-4b31-93a0-bc37a50566c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:42.728971 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ef176-33c5-4b31-93a0-bc37a50566c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:42.729458 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ef176-33c5-4b31-93a0-bc37a50566c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:42.740869 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ef176-33c5-4b31-93a0-bc37a50566c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:42.741954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 948/1910] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:42 2022] POST /identity/v3/users => generated 303 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:42.742782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ef176-33c5-4b31-93a0-bc37a50566c9 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:42.747383 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a952482-9714-4988-9d94-6ecc0a0bd569 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:42.756032 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 963/1911] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:31:42.759499 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a952482-9714-4988-9d94-6ecc0a0bd569 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:42.764647 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259aced2-73ac-4b59-9030-86879e6cc7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:42.765386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259aced2-73ac-4b59-9030-86879e6cc7ed admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:42.765680 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259aced2-73ac-4b59-9030-86879e6cc7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:42.765965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259aced2-73ac-4b59-9030-86879e6cc7ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:42.766707 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259aced2-73ac-4b59-9030-86879e6cc7ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:42.768367 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259aced2-73ac-4b59-9030-86879e6cc7ed admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:42.781629 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 949/1912] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:31:42.788309 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ef176-33c5-4b31-93a0-bc37a50566c9 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:42.800339 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ef176-33c5-4b31-93a0-bc37a50566c9 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:42.808555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67ad64-efb7-40df-8abd-3644ba0cfc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:42.809888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67ad64-efb7-40df-8abd-3644ba0cfc2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:42.810183 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67ad64-efb7-40df-8abd-3644ba0cfc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:42.810685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67ad64-efb7-40df-8abd-3644ba0cfc2e admin admin] PATH_INFO: `/v3/projects/6fdf399d28be49beaf81f071d2a04e30/users/1881b4adcf134a4585c399a7e8c34e83/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:42.830076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67ad64-efb7-40df-8abd-3644ba0cfc2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fdf399d28be49beaf81f071d2a04e30, user_id=1881b4adcf134a4585c399a7e8c34e83, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:42.834420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67ad64-efb7-40df-8abd-3644ba0cfc2e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:42.863184 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 964/1913] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:42 2022] PUT /identity/v3/projects/6fdf399d28be49beaf81f071d2a04e30/users/1881b4adcf134a4585c399a7e8c34e83/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:42.864131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259aced2-73ac-4b59-9030-86879e6cc7ed admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:42.882628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259aced2-73ac-4b59-9030-86879e6cc7ed admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:43.051798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67ad64-efb7-40df-8abd-3644ba0cfc2e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:43.056204 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ce67ad64-efb7-40df-8abd-3644ba0cfc2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:43.139012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2d464-8d7f-4410-bc18-c4bd9b60998e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:43.139012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2d464-8d7f-4410-bc18-c4bd9b60998e admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:43.139012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2d464-8d7f-4410-bc18-c4bd9b60998e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:43.139012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2d464-8d7f-4410-bc18-c4bd9b60998e admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:43.147550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2d464-8d7f-4410-bc18-c4bd9b60998e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:43.149893 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2d464-8d7f-4410-bc18-c4bd9b60998e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:43.166050 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 950/1914] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:43.181185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2d464-8d7f-4410-bc18-c4bd9b60998e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:43.197998 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67ad64-efb7-40df-8abd-3644ba0cfc2e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:43.203918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0e84c-e9bb-4e24-8abc-937d2099b9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:43.205052 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0e84c-e9bb-4e24-8abc-937d2099b9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:43.205485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0e84c-e9bb-4e24-8abc-937d2099b9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:43.205870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0e84c-e9bb-4e24-8abc-937d2099b9d5 admin admin] PATH_INFO: `/v3/projects/e636eba5969b4bcfbccff936b37d5049` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:43.207340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0e84c-e9bb-4e24-8abc-937d2099b9d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e636eba5969b4bcfbccff936b37d5049)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:43.211683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0e84c-e9bb-4e24-8abc-937d2099b9d5 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:43.221328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2d464-8d7f-4410-bc18-c4bd9b60998e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:43.263881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:43.271649 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4dc6e4-e008-421f-9624-6e2164855c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:43.273312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dc6e4-e008-421f-9624-6e2164855c98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:43.276290 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dc6e4-e008-421f-9624-6e2164855c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:43.276842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dc6e4-e008-421f-9624-6e2164855c98 admin admin] PATH_INFO: `/v3/projects/6fdf399d28be49beaf81f071d2a04e30/users/1881b4adcf134a4585c399a7e8c34e83/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:43.293614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-55e0e84c-e9bb-4e24-8abc-937d2099b9d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e636eba5969b4bcfbccff936b37d5049', 'request_id': 'req-55e0e84c-e9bb-4e24-8abc-937d2099b9d5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:43.296422 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4dc6e4-e008-421f-9624-6e2164855c98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fdf399d28be49beaf81f071d2a04e30, user_id=1881b4adcf134a4585c399a7e8c34e83, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:43.301587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-55e0e84c-e9bb-4e24-8abc-937d2099b9d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e636eba5969b4bcfbccff936b37d5049', 'request_id': 'req-55e0e84c-e9bb-4e24-8abc-937d2099b9d5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:43.313428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4dc6e4-e008-421f-9624-6e2164855c98 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:43.359791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-55e0e84c-e9bb-4e24-8abc-937d2099b9d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e636eba5969b4bcfbccff936b37d5049', 'request_id': 'req-55e0e84c-e9bb-4e24-8abc-937d2099b9d5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:43.393141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 951/1915] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:43 2022] PUT /identity/v3/projects/6fdf399d28be49beaf81f071d2a04e30/users/1881b4adcf134a4585c399a7e8c34e83/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:43.399733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f666386-8f29-476b-bddc-fcbf91abbf89 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:43.400312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f666386-8f29-476b-bddc-fcbf91abbf89 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:43.400759 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f666386-8f29-476b-bddc-fcbf91abbf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:43.413036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-55e0e84c-e9bb-4e24-8abc-937d2099b9d5 admin admin] The token cache is being invalidate because project e636eba5969b4bcfbccff936b37d5049 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:43.414876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-55e0e84c-e9bb-4e24-8abc-937d2099b9d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:43.417545 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-55e0e84c-e9bb-4e24-8abc-937d2099b9d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:43.421659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:43.426342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 965/1916] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:43 2022] DELETE /identity/v3/projects/e636eba5969b4bcfbccff936b37d5049 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:43.440079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17a875-790e-436d-9443-08ff57a001e1 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:43.440384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17a875-790e-436d-9443-08ff57a001e1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:43.441062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17a875-790e-436d-9443-08ff57a001e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:43.478797 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1f666386-8f29-476b-bddc-fcbf91abbf89 None None] MFA Rules not processed for user `1881b4adcf134a4585c399a7e8c34e83`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:43.480642 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1f666386-8f29-476b-bddc-fcbf91abbf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:43.492966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3c17a875-790e-436d-9443-08ff57a001e1 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:43.495747 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3c17a875-790e-436d-9443-08ff57a001e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:43.703520 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 952/1917] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:31:43.722976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f666386-8f29-476b-bddc-fcbf91abbf89 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:43.725766 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1f666386-8f29-476b-bddc-fcbf91abbf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:43.767443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 966/1918] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:43.775032 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2c3e1-7611-4536-8b94-e7089c16b0fb None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:43.775566 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2c3e1-7611-4536-8b94-e7089c16b0fb None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:43.776075 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2c3e1-7611-4536-8b94-e7089c16b0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:43.845235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-dea2c3e1-7611-4536-8b94-e7089c16b0fb None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:43.847487 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-dea2c3e1-7611-4536-8b94-e7089c16b0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:43.885241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 967/1919] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:43.890791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2c3e1-7611-4536-8b94-e7089c16b0fb None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:43.909759 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f666386-8f29-476b-bddc-fcbf91abbf89 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:43.918773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2c3e1-7611-4536-8b94-e7089c16b0fb None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:43.924124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfba82-89a5-4749-be95-5c4793c6ae13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:43.928626 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130a352-69f5-43c5-bb34-dc3de313ed31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:43.929766 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130a352-69f5-43c5-bb34-dc3de313ed31 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:43.930220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130a352-69f5-43c5-bb34-dc3de313ed31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:43.930668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130a352-69f5-43c5-bb34-dc3de313ed31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:43.931271 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfba82-89a5-4749-be95-5c4793c6ae13 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:43.931973 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfba82-89a5-4749-be95-5c4793c6ae13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:43.932186 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130a352-69f5-43c5-bb34-dc3de313ed31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:31:43.932483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfba82-89a5-4749-be95-5c4793c6ae13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:43.932753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130a352-69f5-43c5-bb34-dc3de313ed31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:43.936795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130a352-69f5-43c5-bb34-dc3de313ed31 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:43.976210 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 968/1920] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:31:43.983639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bfba82-89a5-4749-be95-5c4793c6ae13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:43.986317 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130a352-69f5-43c5-bb34-dc3de313ed31 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:43.987614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bfba82-89a5-4749-be95-5c4793c6ae13 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:43.990999 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5130a352-69f5-43c5-bb34-dc3de313ed31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:44.022194 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 953/1921] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:43 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:31:44.024689 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfba82-89a5-4749-be95-5c4793c6ae13 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:44.032563 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130a352-69f5-43c5-bb34-dc3de313ed31 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:44.059456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfba82-89a5-4749-be95-5c4793c6ae13 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:44.064320 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:44.066919 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:44.067515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:44.070733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835 admin admin] PATH_INFO: `/v3/projects/7925a9c57cdc4f9b99097fb33a00febb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:44.075260 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91421615-6f83-4e59-ba94-0282f8844550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:44.076045 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7925a9c57cdc4f9b99097fb33a00febb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:44.076419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91421615-6f83-4e59-ba94-0282f8844550 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:44.076771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91421615-6f83-4e59-ba94-0282f8844550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:44.077075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91421615-6f83-4e59-ba94-0282f8844550 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:44.077831 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91421615-6f83-4e59-ba94-0282f8844550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:44.080710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:44.083284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91421615-6f83-4e59-ba94-0282f8844550 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:44.144728 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 954/1922] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:44 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:44.148401 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91421615-6f83-4e59-ba94-0282f8844550 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:44.307378 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91421615-6f83-4e59-ba94-0282f8844550 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:44.311210 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7925a9c57cdc4f9b99097fb33a00febb', 'request_id': 'req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:44.328741 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7925a9c57cdc4f9b99097fb33a00febb', 'request_id': 'req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:44.342735 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:44.363513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7925a9c57cdc4f9b99097fb33a00febb', 'request_id': 'req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:44.403574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835 admin admin] The token cache is being invalidate because project 7925a9c57cdc4f9b99097fb33a00febb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:44.403944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:44.407328 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:44.413459 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 969/1923] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:43 2022] DELETE /identity/v3/projects/7925a9c57cdc4f9b99097fb33a00febb => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:44.415314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:44.419964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:44.424758 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20211324-6163-42b6-868c-98fb41c027bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:44.425507 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20211324-6163-42b6-868c-98fb41c027bf admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:44.425798 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20211324-6163-42b6-868c-98fb41c027bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:44.426078 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20211324-6163-42b6-868c-98fb41c027bf admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:44.430555 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20211324-6163-42b6-868c-98fb41c027bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:44.432986 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20211324-6163-42b6-868c-98fb41c027bf admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:44.487035 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 955/1924] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:44 2022] POST /identity/v3/users => generated 292 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:44.490306 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20211324-6163-42b6-868c-98fb41c027bf admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:44.490560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:44.495448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-20211324-6163-42b6-868c-98fb41c027bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:44.527792 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20211324-6163-42b6-868c-98fb41c027bf admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:44.546888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6c615-9ecd-4dbc-870f-e3edceccaa72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:44.548040 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c615-9ecd-4dbc-870f-e3edceccaa72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:44.548533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c615-9ecd-4dbc-870f-e3edceccaa72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:44.548820 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c615-9ecd-4dbc-870f-e3edceccaa72 admin admin] PATH_INFO: `/v3/users/98a85d1e9029467b9f2b0376eaa3e519` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:44.554839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6c615-9ecd-4dbc-870f-e3edceccaa72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98a85d1e9029467b9f2b0376eaa3e519)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:44.558820 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6c615-9ecd-4dbc-870f-e3edceccaa72 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:44.596586 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e360eb9-14a5-45ef-b5d3-e7b12c98e835 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:44.603585 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c13849-dfbe-4850-aacc-b6ce5b4dddcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:44.604272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c13849-dfbe-4850-aacc-b6ce5b4dddcd service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:44.604561 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c13849-dfbe-4850-aacc-b6ce5b4dddcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:44.604842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c13849-dfbe-4850-aacc-b6ce5b4dddcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:44.607207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a4c13849-dfbe-4850-aacc-b6ce5b4dddcd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:44.667327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c4e6c615-9ecd-4dbc-870f-e3edceccaa72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98a85d1e9029467b9f2b0376eaa3e519'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:44.689197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c4e6c615-9ecd-4dbc-870f-e3edceccaa72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98a85d1e9029467b9f2b0376eaa3e519'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:44.701093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c4e6c615-9ecd-4dbc-870f-e3edceccaa72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98a85d1e9029467b9f2b0376eaa3e519'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:44.702619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c4e6c615-9ecd-4dbc-870f-e3edceccaa72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98a85d1e9029467b9f2b0376eaa3e519'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:44.703225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c4e6c615-9ecd-4dbc-870f-e3edceccaa72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98a85d1e9029467b9f2b0376eaa3e519'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:44.722363 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 956/1925] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:44 2022] DELETE /identity/v3/users/98a85d1e9029467b9f2b0376eaa3e519 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:44.723353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6c615-9ecd-4dbc-870f-e3edceccaa72 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:44.725765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c4e6c615-9ecd-4dbc-870f-e3edceccaa72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:44.728346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c13849-dfbe-4850-aacc-b6ce5b4dddcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:44.730174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c13849-dfbe-4850-aacc-b6ce5b4dddcd service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:44.744706 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 970/1926] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:44.815110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6c615-9ecd-4dbc-870f-e3edceccaa72 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:44.822880 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfc113-5073-4611-8a08-75d1228f7b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:44.824688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfc113-5073-4611-8a08-75d1228f7b10 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:44.824984 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfc113-5073-4611-8a08-75d1228f7b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:44.825394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfc113-5073-4611-8a08-75d1228f7b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:44.826241 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfc113-5073-4611-8a08-75d1228f7b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:44.828878 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfc113-5073-4611-8a08-75d1228f7b10 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:44.870862 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 957/1927] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:44.878354 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c13849-dfbe-4850-aacc-b6ce5b4dddcd service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:44.880287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a4c13849-dfbe-4850-aacc-b6ce5b4dddcd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:44.931431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c13849-dfbe-4850-aacc-b6ce5b4dddcd service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:44.951888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbac30-be89-4f1a-96be-8d4d424c6fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:44.953345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbac30-be89-4f1a-96be-8d4d424c6fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:44.953345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbac30-be89-4f1a-96be-8d4d424c6fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:44.954760 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbac30-be89-4f1a-96be-8d4d424c6fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:44.957050 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cbac30-be89-4f1a-96be-8d4d424c6fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:44.961767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cbac30-be89-4f1a-96be-8d4d424c6fd7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:44.995938 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 971/1928] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:45.000263 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfc113-5073-4611-8a08-75d1228f7b10 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:45.041590 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfc113-5073-4611-8a08-75d1228f7b10 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:45.071304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed183147-72e6-4976-bfbc-2470fa7eb17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:45.080253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed183147-72e6-4976-bfbc-2470fa7eb17a admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:45.083808 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed183147-72e6-4976-bfbc-2470fa7eb17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:45.087104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed183147-72e6-4976-bfbc-2470fa7eb17a admin admin] PATH_INFO: `/v3/projects/e9270d02dc1f48eab107faf94f64c4dd/users/99efebbb2a3e4f32b0a4ed87b1509c32/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:45.174072 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed183147-72e6-4976-bfbc-2470fa7eb17a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9270d02dc1f48eab107faf94f64c4dd, user_id=99efebbb2a3e4f32b0a4ed87b1509c32, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:45.181832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed183147-72e6-4976-bfbc-2470fa7eb17a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:45.197634 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:45.250247 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 958/1929] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:44 2022] PUT /identity/v3/projects/e9270d02dc1f48eab107faf94f64c4dd/users/99efebbb2a3e4f32b0a4ed87b1509c32/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:31:45.265687 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b596c-c3cb-4e1d-94a2-f076741ac82f None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:45.266342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b596c-c3cb-4e1d-94a2-f076741ac82f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:45.266853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b596c-c3cb-4e1d-94a2-f076741ac82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:45.312573 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-628b596c-c3cb-4e1d-94a2-f076741ac82f None None] MFA Rules not processed for user `99efebbb2a3e4f32b0a4ed87b1509c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:45.313993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-628b596c-c3cb-4e1d-94a2-f076741ac82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:45.568757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed183147-72e6-4976-bfbc-2470fa7eb17a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:45.572946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 972/1930] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:45.729237 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b596c-c3cb-4e1d-94a2-f076741ac82f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:45.733344 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-628b596c-c3cb-4e1d-94a2-f076741ac82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:45.806647 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed183147-72e6-4976-bfbc-2470fa7eb17a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:45.818769 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74861c96-254f-45a6-adfe-0af9b3e7b02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:45.820013 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74861c96-254f-45a6-adfe-0af9b3e7b02f admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:45.820603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74861c96-254f-45a6-adfe-0af9b3e7b02f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:45.821208 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74861c96-254f-45a6-adfe-0af9b3e7b02f admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:45.822947 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74861c96-254f-45a6-adfe-0af9b3e7b02f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:45.827514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74861c96-254f-45a6-adfe-0af9b3e7b02f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:45.918833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:45.947344 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 959/1931] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:45 2022] POST /identity/v3/projects => generated 385 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:31:45.948429 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74861c96-254f-45a6-adfe-0af9b3e7b02f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:45.969392 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b596c-c3cb-4e1d-94a2-f076741ac82f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:46.146849 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74861c96-254f-45a6-adfe-0af9b3e7b02f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:46.159479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8443b-739d-4d62-908a-2a90f5815db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:46.162929 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8443b-739d-4d62-908a-2a90f5815db4 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:46.165555 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8443b-739d-4d62-908a-2a90f5815db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:46.166109 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d942dad-b51b-4bf6-bd82-1b64c92c55be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:46.166602 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8443b-739d-4d62-908a-2a90f5815db4 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:46.167402 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d942dad-b51b-4bf6-bd82-1b64c92c55be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:46.167592 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d942dad-b51b-4bf6-bd82-1b64c92c55be admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:46.169653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d942dad-b51b-4bf6-bd82-1b64c92c55be admin admin] PATH_INFO: `/v3/users/ec6e074aae0b409f8b89e10c03a61362` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:46.170711 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d942dad-b51b-4bf6-bd82-1b64c92c55be admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec6e074aae0b409f8b89e10c03a61362)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:46.171219 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8443b-739d-4d62-908a-2a90f5815db4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:46.174476 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8443b-739d-4d62-908a-2a90f5815db4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:46.176828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d942dad-b51b-4bf6-bd82-1b64c92c55be admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:46.212938 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 960/1932] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:45 2022] POST /identity/v3/users => generated 293 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:46.214119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8443b-739d-4d62-908a-2a90f5815db4 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:46.241377 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8443b-739d-4d62-908a-2a90f5815db4 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:46.273273 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b24c1-5f1c-4822-87d2-b9e81ae11437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:46.275368 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b24c1-5f1c-4822-87d2-b9e81ae11437 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:46.276108 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b24c1-5f1c-4822-87d2-b9e81ae11437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:46.277324 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b24c1-5f1c-4822-87d2-b9e81ae11437 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:46.281180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b24c1-5f1c-4822-87d2-b9e81ae11437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:46.285533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b24c1-5f1c-4822-87d2-b9e81ae11437 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:46.311328 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9d942dad-b51b-4bf6-bd82-1b64c92c55be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec6e074aae0b409f8b89e10c03a61362'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:46.312690 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9d942dad-b51b-4bf6-bd82-1b64c92c55be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec6e074aae0b409f8b89e10c03a61362'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:46.319892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 961/1933] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:46.330909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b24c1-5f1c-4822-87d2-b9e81ae11437 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:46.336078 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e84b24c1-5f1c-4822-87d2-b9e81ae11437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:46.376213 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9d942dad-b51b-4bf6-bd82-1b64c92c55be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec6e074aae0b409f8b89e10c03a61362'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:46.379438 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b24c1-5f1c-4822-87d2-b9e81ae11437 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:46.385532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0eba7-27ae-4f56-b3de-7284e8acc266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:46.386190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0eba7-27ae-4f56-b3de-7284e8acc266 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:46.386488 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0eba7-27ae-4f56-b3de-7284e8acc266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:46.386776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0eba7-27ae-4f56-b3de-7284e8acc266 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:46.387707 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0eba7-27ae-4f56-b3de-7284e8acc266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:46.389418 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0eba7-27ae-4f56-b3de-7284e8acc266 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:46.391887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9d942dad-b51b-4bf6-bd82-1b64c92c55be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec6e074aae0b409f8b89e10c03a61362'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:46.406321 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 962/1934] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:46.414294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0eba7-27ae-4f56-b3de-7284e8acc266 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:46.434148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9d942dad-b51b-4bf6-bd82-1b64c92c55be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec6e074aae0b409f8b89e10c03a61362'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:46.445568 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0eba7-27ae-4f56-b3de-7284e8acc266 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:46.451555 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f8a77-6d95-4966-9829-4bf6cfaac46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:46.452788 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f8a77-6d95-4966-9829-4bf6cfaac46a admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:46.453047 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f8a77-6d95-4966-9829-4bf6cfaac46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:46.453527 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f8a77-6d95-4966-9829-4bf6cfaac46a admin admin] PATH_INFO: `/v3/projects/6f2ea80b14c74de7816681e9fc32a9fb/users/f9d7459ec76244e1a881998f9f42c064/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:46.466205 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 973/1935] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:45 2022] DELETE /identity/v3/users/ec6e074aae0b409f8b89e10c03a61362 => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:31:46.469773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa1dd7-583c-4896-9ff0-4aba7c50d258 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:46.470256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa1dd7-583c-4896-9ff0-4aba7c50d258 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:46.470576 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa1dd7-583c-4896-9ff0-4aba7c50d258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:46.487688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f8a77-6d95-4966-9829-4bf6cfaac46a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2ea80b14c74de7816681e9fc32a9fb, user_id=f9d7459ec76244e1a881998f9f42c064, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:46.494145 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f8a77-6d95-4966-9829-4bf6cfaac46a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:46.508601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b8aa1dd7-583c-4896-9ff0-4aba7c50d258 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:46.511460 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b8aa1dd7-583c-4896-9ff0-4aba7c50d258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:46.537695 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 963/1936] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:46 2022] PUT /identity/v3/projects/6f2ea80b14c74de7816681e9fc32a9fb/users/f9d7459ec76244e1a881998f9f42c064/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:31:46.545383 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f8a77-6d95-4966-9829-4bf6cfaac46a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:46.548482 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ea4f8a77-6d95-4966-9829-4bf6cfaac46a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:46.592761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:46.604618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 974/1937] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:46.605946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5005de-3aca-4e8c-92e7-cdece93cc682 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:46.606306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5005de-3aca-4e8c-92e7-cdece93cc682 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:46.606650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5005de-3aca-4e8c-92e7-cdece93cc682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:46.649238 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5c5005de-3aca-4e8c-92e7-cdece93cc682 None None] MFA Rules not processed for user `f9d7459ec76244e1a881998f9f42c064`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:46.650541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5c5005de-3aca-4e8c-92e7-cdece93cc682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:46.662047 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:46.671714 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f8a77-6d95-4966-9829-4bf6cfaac46a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:46.683439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:46.685782 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:46.685782 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:46.686127 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba admin admin] PATH_INFO: `/v3/projects/0ff7e872de7141eb837ce8dc4867a2b1` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:46.687909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ff7e872de7141eb837ce8dc4867a2b1)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:46.690080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:46.790371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 975/1938] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:46.790371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5005de-3aca-4e8c-92e7-cdece93cc682 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:46.790793 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ff7e872de7141eb837ce8dc4867a2b1', 'request_id': 'req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:46.795728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5c5005de-3aca-4e8c-92e7-cdece93cc682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:46.807061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ff7e872de7141eb837ce8dc4867a2b1', 'request_id': 'req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:46.820961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ff7e872de7141eb837ce8dc4867a2b1', 'request_id': 'req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:46.840779 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba admin admin] The token cache is being invalidate because project 0ff7e872de7141eb837ce8dc4867a2b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:46.841107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:46.842037 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:46.846731 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 964/1939] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:46 2022] DELETE /identity/v3/projects/0ff7e872de7141eb837ce8dc4867a2b1 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:46.849687 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:46.851591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:46.984486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf52c2-ffef-4794-b225-7c1c0c2aa6ba admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:46.992419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b7cd33-94ad-4de8-b827-7fcd72b8d695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:46.992673 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5005de-3aca-4e8c-92e7-cdece93cc682 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:46.993514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7cd33-94ad-4de8-b827-7fcd72b8d695 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:46.994026 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7cd33-94ad-4de8-b827-7fcd72b8d695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:46.994506 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7cd33-94ad-4de8-b827-7fcd72b8d695 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:46.995835 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b7cd33-94ad-4de8-b827-7fcd72b8d695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:46.998013 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b7cd33-94ad-4de8-b827-7fcd72b8d695 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:47.056670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 965/1940] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:46 2022] POST /identity/v3/projects => generated 381 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:47.059638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b7cd33-94ad-4de8-b827-7fcd72b8d695 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:47.064181 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-39b7cd33-94ad-4de8-b827-7fcd72b8d695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:47.123729 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b92ecc-1576-44f4-9011-f13c69617c44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:47.124831 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b92ecc-1576-44f4-9011-f13c69617c44 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:47.125613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b92ecc-1576-44f4-9011-f13c69617c44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:47.125977 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b92ecc-1576-44f4-9011-f13c69617c44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:47.128284 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d5b92ecc-1576-44f4-9011-f13c69617c44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:47.143141 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b92ecc-1576-44f4-9011-f13c69617c44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:47.144942 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b92ecc-1576-44f4-9011-f13c69617c44 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:47.166067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 976/1941] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:47.170210 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b92ecc-1576-44f4-9011-f13c69617c44 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:47.188313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:47.205460 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b92ecc-1576-44f4-9011-f13c69617c44 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:47.270356 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b7cd33-94ad-4de8-b827-7fcd72b8d695 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:47.285641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee0beb-967d-4396-99cb-14f850ce56ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:47.287274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee0beb-967d-4396-99cb-14f850ce56ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:47.287653 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee0beb-967d-4396-99cb-14f850ce56ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:47.287947 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee0beb-967d-4396-99cb-14f850ce56ad admin admin] PATH_INFO: `/v3/users/84e453c62d924756adec5bba8994cb80` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:47.293137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee0beb-967d-4396-99cb-14f850ce56ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=84e453c62d924756adec5bba8994cb80)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:47.296273 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee0beb-967d-4396-99cb-14f850ce56ad admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:47.398063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2cad9-0e6e-4914-837b-2d2ce4f0bf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:47.398063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2cad9-0e6e-4914-837b-2d2ce4f0bf05 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:47.398063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2cad9-0e6e-4914-837b-2d2ce4f0bf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:47.398063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2cad9-0e6e-4914-837b-2d2ce4f0bf05 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:47.398985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2cad9-0e6e-4914-837b-2d2ce4f0bf05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:47.398985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2cad9-0e6e-4914-837b-2d2ce4f0bf05 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:47.420605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:47.442544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-15ee0beb-967d-4396-99cb-14f850ce56ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84e453c62d924756adec5bba8994cb80'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:47.443360 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 977/1942] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:47 2022] POST /identity/v3/users => generated 297 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:47.444643 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2cad9-0e6e-4914-837b-2d2ce4f0bf05 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:47.471180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2cad9-0e6e-4914-837b-2d2ce4f0bf05 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:47.479799 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd410d-7a66-45b6-8d77-54bb53eb02e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:47.480059 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-15ee0beb-967d-4396-99cb-14f850ce56ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84e453c62d924756adec5bba8994cb80'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:47.480684 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd410d-7a66-45b6-8d77-54bb53eb02e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:47.480980 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd410d-7a66-45b6-8d77-54bb53eb02e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:47.481265 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd410d-7a66-45b6-8d77-54bb53eb02e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:47.482120 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd410d-7a66-45b6-8d77-54bb53eb02e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:47.484205 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd410d-7a66-45b6-8d77-54bb53eb02e1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:47.499991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-15ee0beb-967d-4396-99cb-14f850ce56ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84e453c62d924756adec5bba8994cb80'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:47.500716 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-15ee0beb-967d-4396-99cb-14f850ce56ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84e453c62d924756adec5bba8994cb80'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:47.501411 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-15ee0beb-967d-4396-99cb-14f850ce56ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84e453c62d924756adec5bba8994cb80'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:47.506745 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 978/1943] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:47.507771 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd410d-7a66-45b6-8d77-54bb53eb02e1 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:47.511378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9fcd410d-7a66-45b6-8d77-54bb53eb02e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:47.524007 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 966/1944] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:47 2022] DELETE /identity/v3/users/84e453c62d924756adec5bba8994cb80 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:47.539671 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd410d-7a66-45b6-8d77-54bb53eb02e1 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:47.688294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eeec48-4c6f-4285-861a-7e361bc0b7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:47.689842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eeec48-4c6f-4285-861a-7e361bc0b7af admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:47.690283 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eeec48-4c6f-4285-861a-7e361bc0b7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:47.690723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eeec48-4c6f-4285-861a-7e361bc0b7af admin admin] PATH_INFO: `/v3/projects/c52a23fb983f4b28b5dae27ae10723a1/users/81b28dc8cd31434497e1264854dc104c/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:47.723433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eeec48-4c6f-4285-861a-7e361bc0b7af admin admin] RBAC: Authorizing `identity:create_grant(project_id=c52a23fb983f4b28b5dae27ae10723a1, user_id=81b28dc8cd31434497e1264854dc104c, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:47.727505 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eeec48-4c6f-4285-861a-7e361bc0b7af admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:47.749523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 979/1945] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:47 2022] PUT /identity/v3/projects/c52a23fb983f4b28b5dae27ae10723a1/users/81b28dc8cd31434497e1264854dc104c/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:47.755568 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee0beb-967d-4396-99cb-14f850ce56ad admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:47.769880 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee0beb-967d-4396-99cb-14f850ce56ad admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:47.871363 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a27f0-8917-44b5-b6a1-f691563177e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:47.872662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a27f0-8917-44b5-b6a1-f691563177e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:47.873001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a27f0-8917-44b5-b6a1-f691563177e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:47.873328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a27f0-8917-44b5-b6a1-f691563177e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:47.876311 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a27f0-8917-44b5-b6a1-f691563177e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:47.879362 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a27f0-8917-44b5-b6a1-f691563177e0 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:47.910744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 967/1946] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:31:47.919231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eeec48-4c6f-4285-861a-7e361bc0b7af admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:47.947634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eeec48-4c6f-4285-861a-7e361bc0b7af admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:47.962345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ab84e-c354-4cbf-a935-bffeb86fe5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:47.964508 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ab84e-c354-4cbf-a935-bffeb86fe5b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:47.965088 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ab84e-c354-4cbf-a935-bffeb86fe5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:47.966055 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ab84e-c354-4cbf-a935-bffeb86fe5b9 admin admin] PATH_INFO: `/v3/projects/c52a23fb983f4b28b5dae27ae10723a1/users/81b28dc8cd31434497e1264854dc104c/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:47.971937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ab84e-c354-4cbf-a935-bffeb86fe5b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c52a23fb983f4b28b5dae27ae10723a1, user_id=81b28dc8cd31434497e1264854dc104c, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:47.994250 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ab84e-c354-4cbf-a935-bffeb86fe5b9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:48.019949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 980/1947] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:47 2022] PUT /identity/v3/projects/c52a23fb983f4b28b5dae27ae10723a1/users/81b28dc8cd31434497e1264854dc104c/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:48.054302 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5e365-378c-4edc-94de-38c172cd854f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:48.056158 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5e365-378c-4edc-94de-38c172cd854f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:48.057028 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5e365-378c-4edc-94de-38c172cd854f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:48.106509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ab84e-c354-4cbf-a935-bffeb86fe5b9 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:48.109907 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e77ab84e-c354-4cbf-a935-bffeb86fe5b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:48.145420 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-38e5e365-378c-4edc-94de-38c172cd854f None None] MFA Rules not processed for user `81b28dc8cd31434497e1264854dc104c`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:48.150536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-38e5e365-378c-4edc-94de-38c172cd854f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:48.250341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:48.318539 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ab84e-c354-4cbf-a935-bffeb86fe5b9 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:48.351988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 968/1948] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:48.352195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5e365-378c-4edc-94de-38c172cd854f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:48.355439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-38e5e365-378c-4edc-94de-38c172cd854f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:48.356013 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:48.357666 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:48.357977 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:48.358479 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76 admin admin] PATH_INFO: `/v3/projects/2535cc942657479c81cbeaa19c3170d7` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:48.361100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2535cc942657479c81cbeaa19c3170d7)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:48.365139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:48.392889 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5e365-378c-4edc-94de-38c172cd854f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:48.408002 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef83912-8890-4263-9174-e22706582a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:48.408002 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef83912-8890-4263-9174-e22706582a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:48.408452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef83912-8890-4263-9174-e22706582a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:48.408647 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef83912-8890-4263-9174-e22706582a55 admin admin] PATH_INFO: `/v3/projects/09b2030a8f4d45d3ac37ff1640c14dcb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:48.413568 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef83912-8890-4263-9174-e22706582a55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09b2030a8f4d45d3ac37ff1640c14dcb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:48.419813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef83912-8890-4263-9174-e22706582a55 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:48.470605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2535cc942657479c81cbeaa19c3170d7', 'request_id': 'req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:48.491958 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2535cc942657479c81cbeaa19c3170d7', 'request_id': 'req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:48.496723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:48.511783 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2535cc942657479c81cbeaa19c3170d7', 'request_id': 'req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:48.528665 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76 admin admin] The token cache is being invalidate because project 2535cc942657479c81cbeaa19c3170d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:48.529366 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:48.530418 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:48.532842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 981/1949] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:48 2022] DELETE /identity/v3/projects/2535cc942657479c81cbeaa19c3170d7 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:48.533860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:48.535289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:48.580927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8ef83912-8890-4263-9174-e22706582a55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09b2030a8f4d45d3ac37ff1640c14dcb', 'request_id': 'req-8ef83912-8890-4263-9174-e22706582a55'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:48.596450 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8ef83912-8890-4263-9174-e22706582a55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09b2030a8f4d45d3ac37ff1640c14dcb', 'request_id': 'req-8ef83912-8890-4263-9174-e22706582a55'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:48.616818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8ef83912-8890-4263-9174-e22706582a55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09b2030a8f4d45d3ac37ff1640c14dcb', 'request_id': 'req-8ef83912-8890-4263-9174-e22706582a55'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:48.638468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f45b01-c93a-45a2-9dc9-3f585b2a8a76 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:48.638972 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8ef83912-8890-4263-9174-e22706582a55 admin admin] The token cache is being invalidate because project 09b2030a8f4d45d3ac37ff1640c14dcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:48.639453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8ef83912-8890-4263-9174-e22706582a55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:48.640937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8ef83912-8890-4263-9174-e22706582a55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:48.648633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 969/1950] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:48 2022] DELETE /identity/v3/projects/09b2030a8f4d45d3ac37ff1640c14dcb => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:48.649833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef83912-8890-4263-9174-e22706582a55 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:48.651457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8ef83912-8890-4263-9174-e22706582a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:48.712874 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f4bbd-1723-4c96-ad5f-5ca4839b4166 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:48.714365 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f4bbd-1723-4c96-ad5f-5ca4839b4166 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:48.714746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f4bbd-1723-4c96-ad5f-5ca4839b4166 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:48.715475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f4bbd-1723-4c96-ad5f-5ca4839b4166 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:48.723055 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ba9f4bbd-1723-4c96-ad5f-5ca4839b4166 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:48.805347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef83912-8890-4263-9174-e22706582a55 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:48.826170 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f7664-0dbe-49a9-9533-4b8f5466cde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:48.827530 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f7664-0dbe-49a9-9533-4b8f5466cde7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:48.827937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f7664-0dbe-49a9-9533-4b8f5466cde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:48.828361 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f7664-0dbe-49a9-9533-4b8f5466cde7 admin admin] PATH_INFO: `/v3/users/bf587de43822495da80c69551d11fd5f` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:48.829922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0f7664-0dbe-49a9-9533-4b8f5466cde7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf587de43822495da80c69551d11fd5f)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:48.832265 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0f7664-0dbe-49a9-9533-4b8f5466cde7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:48.848564 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f4bbd-1723-4c96-ad5f-5ca4839b4166 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:48.851033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f4bbd-1723-4c96-ad5f-5ca4839b4166 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:48.865139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 982/1951] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:48.868231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b15260-e020-4a25-bd37-545f5bff1b44 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:48.868231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b15260-e020-4a25-bd37-545f5bff1b44 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:48.868943 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b15260-e020-4a25-bd37-545f5bff1b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:48.915912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a0b15260-e020-4a25-bd37-545f5bff1b44 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:48.938751 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a0b15260-e020-4a25-bd37-545f5bff1b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:48.964618 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2a0f7664-0dbe-49a9-9533-4b8f5466cde7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf587de43822495da80c69551d11fd5f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:48.985596 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2a0f7664-0dbe-49a9-9533-4b8f5466cde7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf587de43822495da80c69551d11fd5f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:48.985844 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 983/1952] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:48.988515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:49.001242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070bdbd-0a4e-4922-b4ce-4a8dafae1f9e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:49.001931 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070bdbd-0a4e-4922-b4ce-4a8dafae1f9e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:49.002391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070bdbd-0a4e-4922-b4ce-4a8dafae1f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:49.013640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2a0f7664-0dbe-49a9-9533-4b8f5466cde7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf587de43822495da80c69551d11fd5f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:49.015398 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2a0f7664-0dbe-49a9-9533-4b8f5466cde7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf587de43822495da80c69551d11fd5f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:49.016379 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2a0f7664-0dbe-49a9-9533-4b8f5466cde7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf587de43822495da80c69551d11fd5f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:49.046961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 970/1953] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:48 2022] DELETE /identity/v3/users/bf587de43822495da80c69551d11fd5f => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:49.069785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5070bdbd-0a4e-4922-b4ce-4a8dafae1f9e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:49.071501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5070bdbd-0a4e-4922-b4ce-4a8dafae1f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:49.215305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:49.266084 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 984/1954] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:31:49.274005 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f7664-0dbe-49a9-9533-4b8f5466cde7 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:49.352781 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f7664-0dbe-49a9-9533-4b8f5466cde7 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:49.374858 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac7200-ec53-422e-aeac-ebbb5e255043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:49.387220 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac7200-ec53-422e-aeac-ebbb5e255043 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:49.390373 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac7200-ec53-422e-aeac-ebbb5e255043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:49.391713 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac7200-ec53-422e-aeac-ebbb5e255043 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:49.394982 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac7200-ec53-422e-aeac-ebbb5e255043 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:31:49.396698 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac7200-ec53-422e-aeac-ebbb5e255043 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:49.415147 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac7200-ec53-422e-aeac-ebbb5e255043 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:49.449428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 971/1955] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:31:49.455620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070bdbd-0a4e-4922-b4ce-4a8dafae1f9e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:49.487693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070bdbd-0a4e-4922-b4ce-4a8dafae1f9e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:49.523925 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28a995-fd94-4f80-879c-88cbb8d4a11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:49.523925 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28a995-fd94-4f80-879c-88cbb8d4a11b admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:49.524486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28a995-fd94-4f80-879c-88cbb8d4a11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:49.524954 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28a995-fd94-4f80-879c-88cbb8d4a11b admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:49.526397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28a995-fd94-4f80-879c-88cbb8d4a11b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:49.530641 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28a995-fd94-4f80-879c-88cbb8d4a11b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:49.654267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 985/1956] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:49 2022] POST /identity/v3/projects => generated 371 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:49.661862 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac7200-ec53-422e-aeac-ebbb5e255043 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:49.716825 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac7200-ec53-422e-aeac-ebbb5e255043 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:49.972362 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a1956-dc6e-42fb-8727-eaeae4299d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:49.977556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a1956-dc6e-42fb-8727-eaeae4299d55 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:49.980198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a1956-dc6e-42fb-8727-eaeae4299d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:49.980838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a1956-dc6e-42fb-8727-eaeae4299d55 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:49.984367 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a1956-dc6e-42fb-8727-eaeae4299d55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:49.990679 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a1956-dc6e-42fb-8727-eaeae4299d55 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:50.059524 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28a995-fd94-4f80-879c-88cbb8d4a11b admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:50.062688 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ed28a995-fd94-4f80-879c-88cbb8d4a11b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:50.101565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 972/1957] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:49 2022] POST /identity/v3/users => generated 286 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:31:50.110397 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a1956-dc6e-42fb-8727-eaeae4299d55 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:50.114908 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28a995-fd94-4f80-879c-88cbb8d4a11b admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:50.135243 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0967a0-a67d-43a3-a6f0-74698d40ceb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:50.136379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0967a0-a67d-43a3-a6f0-74698d40ceb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:50.136941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0967a0-a67d-43a3-a6f0-74698d40ceb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:50.137443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0967a0-a67d-43a3-a6f0-74698d40ceb4 admin admin] PATH_INFO: `/v3/projects/12b8cb7b8cd440e9a4dea8f1688d31c9` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:50.140186 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0967a0-a67d-43a3-a6f0-74698d40ceb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b8cb7b8cd440e9a4dea8f1688d31c9)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:50.142602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0967a0-a67d-43a3-a6f0-74698d40ceb4 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:50.148389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a1956-dc6e-42fb-8727-eaeae4299d55 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:50.159632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73f6dd-557c-45d5-b60c-2dc37a6477d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:50.163294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73f6dd-557c-45d5-b60c-2dc37a6477d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:50.164784 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73f6dd-557c-45d5-b60c-2dc37a6477d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:50.165581 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73f6dd-557c-45d5-b60c-2dc37a6477d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:50.167771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d73f6dd-557c-45d5-b60c-2dc37a6477d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:50.174451 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d73f6dd-557c-45d5-b60c-2dc37a6477d8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:50.216385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 973/1958] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:50.223224 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73f6dd-557c-45d5-b60c-2dc37a6477d8 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:50.280942 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73f6dd-557c-45d5-b60c-2dc37a6477d8 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:50.296457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0450d-6b32-4ce4-b520-e44a504ca158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:50.297628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0450d-6b32-4ce4-b520-e44a504ca158 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:50.298117 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0450d-6b32-4ce4-b520-e44a504ca158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:50.298596 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0450d-6b32-4ce4-b520-e44a504ca158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:50.306690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0450d-6b32-4ce4-b520-e44a504ca158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:50.316414 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0450d-6b32-4ce4-b520-e44a504ca158 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:50.341714 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-db0967a0-a67d-43a3-a6f0-74698d40ceb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b8cb7b8cd440e9a4dea8f1688d31c9', 'request_id': 'req-db0967a0-a67d-43a3-a6f0-74698d40ceb4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:50.358943 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 974/1959] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:50.358943 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0450d-6b32-4ce4-b520-e44a504ca158 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:50.358943 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cdb0450d-6b32-4ce4-b520-e44a504ca158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:50.359998 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-db0967a0-a67d-43a3-a6f0-74698d40ceb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b8cb7b8cd440e9a4dea8f1688d31c9', 'request_id': 'req-db0967a0-a67d-43a3-a6f0-74698d40ceb4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:50.377798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-db0967a0-a67d-43a3-a6f0-74698d40ceb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b8cb7b8cd440e9a4dea8f1688d31c9', 'request_id': 'req-db0967a0-a67d-43a3-a6f0-74698d40ceb4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:50.381017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0450d-6b32-4ce4-b520-e44a504ca158 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:50.408493 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-db0967a0-a67d-43a3-a6f0-74698d40ceb4 admin admin] The token cache is being invalidate because project 12b8cb7b8cd440e9a4dea8f1688d31c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:50.409424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-db0967a0-a67d-43a3-a6f0-74698d40ceb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:50.411069 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-db0967a0-a67d-43a3-a6f0-74698d40ceb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:50.419983 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 986/1960] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:50 2022] DELETE /identity/v3/projects/12b8cb7b8cd440e9a4dea8f1688d31c9 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:50.424102 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0967a0-a67d-43a3-a6f0-74698d40ceb4 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:50.427423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-db0967a0-a67d-43a3-a6f0-74698d40ceb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:50.550480 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e77af-989c-4292-b741-51e5b199d178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:50.553849 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e77af-989c-4292-b741-51e5b199d178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:50.554476 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e77af-989c-4292-b741-51e5b199d178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:50.555428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e77af-989c-4292-b741-51e5b199d178 admin admin] PATH_INFO: `/v3/users/e34a7e359dff4673b00dced96373fbbb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:50.559905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e77af-989c-4292-b741-51e5b199d178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e34a7e359dff4673b00dced96373fbbb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:50.569254 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0967a0-a67d-43a3-a6f0-74698d40ceb4 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:50.569574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e77af-989c-4292-b741-51e5b199d178 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:50.576020 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b650843-d826-4b76-b82a-9524430b5139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:50.577678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b650843-d826-4b76-b82a-9524430b5139 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:50.578297 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b650843-d826-4b76-b82a-9524430b5139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:50.578781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b650843-d826-4b76-b82a-9524430b5139 admin admin] PATH_INFO: `/v3/projects/a6d61985c0514e699483e2c02324cdef/users/4386014ca63b4976ac6356dcf26b2ce4/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:50.609105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b650843-d826-4b76-b82a-9524430b5139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6d61985c0514e699483e2c02324cdef, user_id=4386014ca63b4976ac6356dcf26b2ce4, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:50.612414 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b650843-d826-4b76-b82a-9524430b5139 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:50.655587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 987/1961] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:50 2022] PUT /identity/v3/projects/a6d61985c0514e699483e2c02324cdef/users/4386014ca63b4976ac6356dcf26b2ce4/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:50.658721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeceeaf-76d9-4355-a034-650db41bbad2 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:50.659499 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeceeaf-76d9-4355-a034-650db41bbad2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:50.660037 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeceeaf-76d9-4355-a034-650db41bbad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:50.670044 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:50.695183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4d5e77af-989c-4292-b741-51e5b199d178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e34a7e359dff4673b00dced96373fbbb'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:50.698476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-2aeceeaf-76d9-4355-a034-650db41bbad2 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:50.701177 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2aeceeaf-76d9-4355-a034-650db41bbad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:50.728446 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4d5e77af-989c-4292-b741-51e5b199d178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e34a7e359dff4673b00dced96373fbbb'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:50.747317 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4d5e77af-989c-4292-b741-51e5b199d178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e34a7e359dff4673b00dced96373fbbb'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:50.749036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4d5e77af-989c-4292-b741-51e5b199d178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e34a7e359dff4673b00dced96373fbbb'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:50.751513 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4d5e77af-989c-4292-b741-51e5b199d178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e34a7e359dff4673b00dced96373fbbb'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:50.776397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:50.790220 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 975/1962] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:50 2022] DELETE /identity/v3/users/e34a7e359dff4673b00dced96373fbbb => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:50.794582 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5639fd-cb51-426a-9639-cb87cf2c2435 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:50.795909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5639fd-cb51-426a-9639-cb87cf2c2435 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:50.797411 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5639fd-cb51-426a-9639-cb87cf2c2435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:50.841694 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-6b5639fd-cb51-426a-9639-cb87cf2c2435 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:50.845682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6b5639fd-cb51-426a-9639-cb87cf2c2435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:50.878143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 988/1963] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:50.878143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47313cbe-5f99-440b-b095-ed869c6ef019 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:50.878143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47313cbe-5f99-440b-b095-ed869c6ef019 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:50.878143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47313cbe-5f99-440b-b095-ed869c6ef019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:50.905371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-47313cbe-5f99-440b-b095-ed869c6ef019 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:50.906753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-47313cbe-5f99-440b-b095-ed869c6ef019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:51.060696 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 976/1964] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:51.063100 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf765e2-ef42-4d6d-84d7-48a8f664dd4a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:51.063397 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf765e2-ef42-4d6d-84d7-48a8f664dd4a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:51.063690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf765e2-ef42-4d6d-84d7-48a8f664dd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:51.070948 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 989/1965] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:51.072242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47313cbe-5f99-440b-b095-ed869c6ef019 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:51.074013 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-47313cbe-5f99-440b-b095-ed869c6ef019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:51.114284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9bf765e2-ef42-4d6d-84d7-48a8f664dd4a None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:51.118369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9bf765e2-ef42-4d6d-84d7-48a8f664dd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:51.141217 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 977/1966] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:51.146585 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf765e2-ef42-4d6d-84d7-48a8f664dd4a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:51.150458 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9bf765e2-ef42-4d6d-84d7-48a8f664dd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:51.169553 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47313cbe-5f99-440b-b095-ed869c6ef019 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:51.180294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf765e2-ef42-4d6d-84d7-48a8f664dd4a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:51.198312 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7a7ad-ed9e-4d95-b65e-ccf2485aaf35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:51.198596 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796eca2f-e150-497a-ba59-ec4cb088278a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:51.199054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796eca2f-e150-497a-ba59-ec4cb088278a service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:51.199376 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796eca2f-e150-497a-ba59-ec4cb088278a service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:51.199615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7a7ad-ed9e-4d95-b65e-ccf2485aaf35 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:51.199781 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796eca2f-e150-497a-ba59-ec4cb088278a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:51.200003 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7a7ad-ed9e-4d95-b65e-ccf2485aaf35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:51.200341 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7a7ad-ed9e-4d95-b65e-ccf2485aaf35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:51.219281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796eca2f-e150-497a-ba59-ec4cb088278a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:51.220928 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796eca2f-e150-497a-ba59-ec4cb088278a service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:51.221713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7a7ad-ed9e-4d95-b65e-ccf2485aaf35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:51.226390 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7a7ad-ed9e-4d95-b65e-ccf2485aaf35 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:51.246645 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 978/1967] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:51.249436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796eca2f-e150-497a-ba59-ec4cb088278a service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:51.256169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 990/1968] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:51.277460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796eca2f-e150-497a-ba59-ec4cb088278a service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:51.293077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea686299-c7f9-474d-bad2-c049e90d60ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:51.296008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea686299-c7f9-474d-bad2-c049e90d60ec admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:51.296008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea686299-c7f9-474d-bad2-c049e90d60ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:51.296008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea686299-c7f9-474d-bad2-c049e90d60ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:51.296008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea686299-c7f9-474d-bad2-c049e90d60ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:31:51.296008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea686299-c7f9-474d-bad2-c049e90d60ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:51.297035 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea686299-c7f9-474d-bad2-c049e90d60ec admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:51.319356 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 979/1969] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:31:51.324218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7a7ad-ed9e-4d95-b65e-ccf2485aaf35 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:51.336613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7a7ad-ed9e-4d95-b65e-ccf2485aaf35 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:51.346187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a9ec7-0924-4330-be4c-958cf32cb92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:51.346881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a9ec7-0924-4330-be4c-958cf32cb92a admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:51.347169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a9ec7-0924-4330-be4c-958cf32cb92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:51.347447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a9ec7-0924-4330-be4c-958cf32cb92a admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:51.348420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a9ec7-0924-4330-be4c-958cf32cb92a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:51.351799 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a9ec7-0924-4330-be4c-958cf32cb92a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:51.394807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea686299-c7f9-474d-bad2-c049e90d60ec admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:51.398454 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ea686299-c7f9-474d-bad2-c049e90d60ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:51.422980 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 991/1970] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:51 2022] POST /identity/v3/projects => generated 399 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:51.430263 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a9ec7-0924-4330-be4c-958cf32cb92a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:51.526218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea686299-c7f9-474d-bad2-c049e90d60ec admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:51.573543 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:51.579543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:51.584774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a9ec7-0924-4330-be4c-958cf32cb92a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:51.592197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80334b86-de5f-4941-aad5-b1161fbe0186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:51.598480 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80334b86-de5f-4941-aad5-b1161fbe0186 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:51.600439 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80334b86-de5f-4941-aad5-b1161fbe0186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:51.600743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80334b86-de5f-4941-aad5-b1161fbe0186 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:51.601571 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80334b86-de5f-4941-aad5-b1161fbe0186 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:51.606953 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80334b86-de5f-4941-aad5-b1161fbe0186 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:51.607912 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f15495-3ab5-4b37-b7b9-73a70ac87df3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:51.608862 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f15495-3ab5-4b37-b7b9-73a70ac87df3 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:51.609228 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f15495-3ab5-4b37-b7b9-73a70ac87df3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:51.609621 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f15495-3ab5-4b37-b7b9-73a70ac87df3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:51.612178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f5f15495-3ab5-4b37-b7b9-73a70ac87df3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:51.652518 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 992/1971] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:51 2022] POST /identity/v3/users => generated 300 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:31:51.656631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80334b86-de5f-4941-aad5-b1161fbe0186 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:51.670555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80334b86-de5f-4941-aad5-b1161fbe0186 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:51.678867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489ba9e-cab3-48f8-9225-dc834ea5f44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:51.679731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489ba9e-cab3-48f8-9225-dc834ea5f44f admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:51.680010 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489ba9e-cab3-48f8-9225-dc834ea5f44f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:51.680296 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489ba9e-cab3-48f8-9225-dc834ea5f44f admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:51.681200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489ba9e-cab3-48f8-9225-dc834ea5f44f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:51.683091 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489ba9e-cab3-48f8-9225-dc834ea5f44f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:51.697048 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 993/1972] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:31:51.709093 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489ba9e-cab3-48f8-9225-dc834ea5f44f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:51.725555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489ba9e-cab3-48f8-9225-dc834ea5f44f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:51.731196 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0007d-f157-483f-83eb-5a1eaee95f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:51.731891 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0007d-f157-483f-83eb-5a1eaee95f5d admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:51.732170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0007d-f157-483f-83eb-5a1eaee95f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:51.732449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0007d-f157-483f-83eb-5a1eaee95f5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:51.733200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0007d-f157-483f-83eb-5a1eaee95f5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:51.735118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0007d-f157-483f-83eb-5a1eaee95f5d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:51.748779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 994/1973] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:51.754569 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0007d-f157-483f-83eb-5a1eaee95f5d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:51.784508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f15495-3ab5-4b37-b7b9-73a70ac87df3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:51.790939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f15495-3ab5-4b37-b7b9-73a70ac87df3 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:51.794584 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0007d-f157-483f-83eb-5a1eaee95f5d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:51.812262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782fe02-21f2-4d59-87c3-cf58ff7c69a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:51.813190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782fe02-21f2-4d59-87c3-cf58ff7c69a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:51.813501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782fe02-21f2-4d59-87c3-cf58ff7c69a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:51.813782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782fe02-21f2-4d59-87c3-cf58ff7c69a6 admin admin] PATH_INFO: `/v3/projects/f133e7b0810e431ba84313a672c5ace9/users/1e94688cd938492da8a03259f995fc17/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:51.815915 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 980/1974] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:31:51.853079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782fe02-21f2-4d59-87c3-cf58ff7c69a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f133e7b0810e431ba84313a672c5ace9, user_id=1e94688cd938492da8a03259f995fc17, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:51.858207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782fe02-21f2-4d59-87c3-cf58ff7c69a6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:51.900410 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 995/1975] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:51 2022] PUT /identity/v3/projects/f133e7b0810e431ba84313a672c5ace9/users/1e94688cd938492da8a03259f995fc17/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:51.924083 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae0078-5244-4fbc-9188-0d30b06b7f47 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:51.924564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae0078-5244-4fbc-9188-0d30b06b7f47 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:51.925272 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae0078-5244-4fbc-9188-0d30b06b7f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:51.988642 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-8cae0078-5244-4fbc-9188-0d30b06b7f47 None None] MFA Rules not processed for user `1e94688cd938492da8a03259f995fc17`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:51.996402 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8cae0078-5244-4fbc-9188-0d30b06b7f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:52.032808 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782fe02-21f2-4d59-87c3-cf58ff7c69a6 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:52.038702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d782fe02-21f2-4d59-87c3-cf58ff7c69a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:52.185969 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782fe02-21f2-4d59-87c3-cf58ff7c69a6 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:52.204143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 981/1976] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:52.206152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455057ea-9d40-4f9f-b06e-75e3b73c3500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:52.207224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455057ea-9d40-4f9f-b06e-75e3b73c3500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:52.207490 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae0078-5244-4fbc-9188-0d30b06b7f47 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:52.207601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455057ea-9d40-4f9f-b06e-75e3b73c3500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:52.207781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455057ea-9d40-4f9f-b06e-75e3b73c3500 admin admin] PATH_INFO: `/v3/projects/d9166848cb5e4137bcfe3c5acc26882e` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:52.210030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455057ea-9d40-4f9f-b06e-75e3b73c3500 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9166848cb5e4137bcfe3c5acc26882e)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:52.218154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455057ea-9d40-4f9f-b06e-75e3b73c3500 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:52.245939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae0078-5244-4fbc-9188-0d30b06b7f47 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:52.263700 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca5cfe-1999-4a07-a35f-702b12817e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:52.265389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca5cfe-1999-4a07-a35f-702b12817e2f admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:52.265805 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca5cfe-1999-4a07-a35f-702b12817e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:52.266250 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca5cfe-1999-4a07-a35f-702b12817e2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:52.268356 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca5cfe-1999-4a07-a35f-702b12817e2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:52.285207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca5cfe-1999-4a07-a35f-702b12817e2f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:52.386408 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 982/1977] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:52 2022] POST /identity/v3/projects => generated 399 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:52.398422 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca5cfe-1999-4a07-a35f-702b12817e2f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:52.538878 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-455057ea-9d40-4f9f-b06e-75e3b73c3500 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9166848cb5e4137bcfe3c5acc26882e', 'request_id': 'req-455057ea-9d40-4f9f-b06e-75e3b73c3500'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:52.561841 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-455057ea-9d40-4f9f-b06e-75e3b73c3500 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9166848cb5e4137bcfe3c5acc26882e', 'request_id': 'req-455057ea-9d40-4f9f-b06e-75e3b73c3500'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:52.610909 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-455057ea-9d40-4f9f-b06e-75e3b73c3500 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9166848cb5e4137bcfe3c5acc26882e', 'request_id': 'req-455057ea-9d40-4f9f-b06e-75e3b73c3500'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:52.619717 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca5cfe-1999-4a07-a35f-702b12817e2f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:52.632640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:52.668517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-455057ea-9d40-4f9f-b06e-75e3b73c3500 admin admin] The token cache is being invalidate because project d9166848cb5e4137bcfe3c5acc26882e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:52.668893 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-455057ea-9d40-4f9f-b06e-75e3b73c3500 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:52.671889 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-455057ea-9d40-4f9f-b06e-75e3b73c3500 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:52.675373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 996/1978] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:52 2022] DELETE /identity/v3/projects/d9166848cb5e4137bcfe3c5acc26882e => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:31:52.684527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455057ea-9d40-4f9f-b06e-75e3b73c3500 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:52.691281 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-455057ea-9d40-4f9f-b06e-75e3b73c3500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:52.738444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb02ed4-750f-4769-af99-ca1f5e6b0850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:52.739419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb02ed4-750f-4769-af99-ca1f5e6b0850 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:52.739903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb02ed4-750f-4769-af99-ca1f5e6b0850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:52.740692 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb02ed4-750f-4769-af99-ca1f5e6b0850 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:52.741792 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb02ed4-750f-4769-af99-ca1f5e6b0850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:52.744136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb02ed4-750f-4769-af99-ca1f5e6b0850 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:52.783802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:52.811671 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 983/1979] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:52 2022] POST /identity/v3/users => generated 306 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:52.812695 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb02ed4-750f-4769-af99-ca1f5e6b0850 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:52.824421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-adb02ed4-750f-4769-af99-ca1f5e6b0850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:52.858510 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb02ed4-750f-4769-af99-ca1f5e6b0850 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:52.866392 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455057ea-9d40-4f9f-b06e-75e3b73c3500 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:52.869883 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1ac87-2f96-4d73-858a-e04b547d958b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:52.872014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1ac87-2f96-4d73-858a-e04b547d958b admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:52.872014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1ac87-2f96-4d73-858a-e04b547d958b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:52.872014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1ac87-2f96-4d73-858a-e04b547d958b admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:52.872485 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b1ac87-2f96-4d73-858a-e04b547d958b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:52.875744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b1ac87-2f96-4d73-858a-e04b547d958b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:52.884004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c758e6e-9ce8-4999-b707-620fad43d29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:52.885033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c758e6e-9ce8-4999-b707-620fad43d29b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:52.885341 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c758e6e-9ce8-4999-b707-620fad43d29b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:52.885628 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c758e6e-9ce8-4999-b707-620fad43d29b admin admin] PATH_INFO: `/v3/users/1881b4adcf134a4585c399a7e8c34e83` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:52.889994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c758e6e-9ce8-4999-b707-620fad43d29b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1881b4adcf134a4585c399a7e8c34e83)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:52.893629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c758e6e-9ce8-4999-b707-620fad43d29b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:52.910047 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 984/1980] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:52.912423 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1ac87-2f96-4d73-858a-e04b547d958b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:52.939998 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1ac87-2f96-4d73-858a-e04b547d958b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:52.956232 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13026d4-8a83-458b-b98d-535d09e57db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:52.957343 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13026d4-8a83-458b-b98d-535d09e57db1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:52.957642 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13026d4-8a83-458b-b98d-535d09e57db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:52.960268 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13026d4-8a83-458b-b98d-535d09e57db1 admin admin] PATH_INFO: `/v3/projects/5ad49bbe963c4ac5b317ef7ea7d5452c/users/583cbec012e340328de15714f649bb13/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:52.992716 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13026d4-8a83-458b-b98d-535d09e57db1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad49bbe963c4ac5b317ef7ea7d5452c, user_id=583cbec012e340328de15714f649bb13, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:53.003867 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13026d4-8a83-458b-b98d-535d09e57db1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:53.019206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3c758e6e-9ce8-4999-b707-620fad43d29b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1881b4adcf134a4585c399a7e8c34e83'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:53.019988 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3c758e6e-9ce8-4999-b707-620fad43d29b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1881b4adcf134a4585c399a7e8c34e83'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:53.041729 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 985/1981] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:52 2022] PUT /identity/v3/projects/5ad49bbe963c4ac5b317ef7ea7d5452c/users/583cbec012e340328de15714f649bb13/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:53.046162 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13026d4-8a83-458b-b98d-535d09e57db1 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:53.051526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a13026d4-8a83-458b-b98d-535d09e57db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:53.065861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3c758e6e-9ce8-4999-b707-620fad43d29b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1881b4adcf134a4585c399a7e8c34e83'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:53.071285 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3c758e6e-9ce8-4999-b707-620fad43d29b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1881b4adcf134a4585c399a7e8c34e83'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:53.085963 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3c758e6e-9ce8-4999-b707-620fad43d29b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1881b4adcf134a4585c399a7e8c34e83'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:53.125834 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 997/1982] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:52 2022] DELETE /identity/v3/users/1881b4adcf134a4585c399a7e8c34e83 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:53.252509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13026d4-8a83-458b-b98d-535d09e57db1 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:53.269025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c758e6e-9ce8-4999-b707-620fad43d29b admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:53.271080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3c758e6e-9ce8-4999-b707-620fad43d29b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:53.455953 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8a3a2-afe5-4dfd-853d-2b16967c21fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:53.456678 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8a3a2-afe5-4dfd-853d-2b16967c21fc admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:53.456976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8a3a2-afe5-4dfd-853d-2b16967c21fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:53.457261 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8a3a2-afe5-4dfd-853d-2b16967c21fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:53.458642 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8a3a2-afe5-4dfd-853d-2b16967c21fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:53.461672 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8a3a2-afe5-4dfd-853d-2b16967c21fc admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:53.474667 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c758e6e-9ce8-4999-b707-620fad43d29b admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:53.481700 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 986/1983] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:53.482370 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8a3a2-afe5-4dfd-853d-2b16967c21fc admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:53.486642 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132f8ed-a192-483c-b3cf-7e62c4f54a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:53.491693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132f8ed-a192-483c-b3cf-7e62c4f54a64 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:53.496758 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132f8ed-a192-483c-b3cf-7e62c4f54a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:53.497511 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132f8ed-a192-483c-b3cf-7e62c4f54a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:53.498975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:53.502601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9132f8ed-a192-483c-b3cf-7e62c4f54a64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:53.503915 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8a3a2-afe5-4dfd-853d-2b16967c21fc admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:53.524150 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ff058-7a18-4d8e-8cb6-9ab0f89284f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:53.525099 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ff058-7a18-4d8e-8cb6-9ab0f89284f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:53.526081 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ff058-7a18-4d8e-8cb6-9ab0f89284f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:53.526411 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ff058-7a18-4d8e-8cb6-9ab0f89284f9 admin admin] PATH_INFO: `/v3/projects/5ad49bbe963c4ac5b317ef7ea7d5452c/users/583cbec012e340328de15714f649bb13/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:53.534987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ff058-7a18-4d8e-8cb6-9ab0f89284f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad49bbe963c4ac5b317ef7ea7d5452c, user_id=583cbec012e340328de15714f649bb13, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:53.539081 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ff058-7a18-4d8e-8cb6-9ab0f89284f9 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:53.575176 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 987/1984] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:53 2022] PUT /identity/v3/projects/5ad49bbe963c4ac5b317ef7ea7d5452c/users/583cbec012e340328de15714f649bb13/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:53.588763 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba18276-928c-46c4-86b9-41160b94b1a4 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:53.589171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba18276-928c-46c4-86b9-41160b94b1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:53.589426 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba18276-928c-46c4-86b9-41160b94b1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:53.649796 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9ba18276-928c-46c4-86b9-41160b94b1a4 None None] MFA Rules not processed for user `583cbec012e340328de15714f649bb13`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:53.655025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9ba18276-928c-46c4-86b9-41160b94b1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:53.696600 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132f8ed-a192-483c-b3cf-7e62c4f54a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:53.700667 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132f8ed-a192-483c-b3cf-7e62c4f54a64 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:53.713787 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:53.730756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 998/1985] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:31:53.789030 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 988/1986] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:53 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:53.802187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132f8ed-a192-483c-b3cf-7e62c4f54a64 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:53.804175 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9132f8ed-a192-483c-b3cf-7e62c4f54a64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:53.832528 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132f8ed-a192-483c-b3cf-7e62c4f54a64 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:53.936627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb037c-1a0b-4daf-a678-9d9de7d9b1f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:53.938145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb037c-1a0b-4daf-a678-9d9de7d9b1f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:53.938404 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb037c-1a0b-4daf-a678-9d9de7d9b1f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:53.938694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb037c-1a0b-4daf-a678-9d9de7d9b1f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:53.954710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb037c-1a0b-4daf-a678-9d9de7d9b1f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:53.956146 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb037c-1a0b-4daf-a678-9d9de7d9b1f2 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:53.980398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 999/1987] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:53 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:54.118598 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba18276-928c-46c4-86b9-41160b94b1a4 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:54.137074 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba18276-928c-46c4-86b9-41160b94b1a4 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:54.142492 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f64ef5-564e-4555-99ac-38ce1b01a950 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:54.143333 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f64ef5-564e-4555-99ac-38ce1b01a950 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:54.143663 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f64ef5-564e-4555-99ac-38ce1b01a950 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:54.144340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f64ef5-564e-4555-99ac-38ce1b01a950 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:54.147940 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-55f64ef5-564e-4555-99ac-38ce1b01a950 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:54.177910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97100b05-faad-44d5-b83e-4e9e341e8d2e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:54.178253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97100b05-faad-44d5-b83e-4e9e341e8d2e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:54.178612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97100b05-faad-44d5-b83e-4e9e341e8d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:54.283559 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-97100b05-faad-44d5-b83e-4e9e341e8d2e None None] MFA Rules not processed for user `4386014ca63b4976ac6356dcf26b2ce4`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:54.285673 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-97100b05-faad-44d5-b83e-4e9e341e8d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:54.340882 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f64ef5-564e-4555-99ac-38ce1b01a950 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:54.343054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f64ef5-564e-4555-99ac-38ce1b01a950 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:54.366333 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 989/1988] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:54 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:31:54.481638 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1000/1989] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:54.486281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f64ef5-564e-4555-99ac-38ce1b01a950 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:54.487830 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-55f64ef5-564e-4555-99ac-38ce1b01a950 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:54.635217 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f64ef5-564e-4555-99ac-38ce1b01a950 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:54.646279 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df477746-c29f-46dd-9364-c8ab72e488a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:54.646987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df477746-c29f-46dd-9364-c8ab72e488a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:54.647434 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df477746-c29f-46dd-9364-c8ab72e488a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:54.647720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df477746-c29f-46dd-9364-c8ab72e488a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:54.652024 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df477746-c29f-46dd-9364-c8ab72e488a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:54.652024 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df477746-c29f-46dd-9364-c8ab72e488a5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:54.697759 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:54.727075 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97100b05-faad-44d5-b83e-4e9e341e8d2e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:54.737226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 990/1990] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:54 2022] POST /identity/v3/projects => generated 371 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:54.745887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97100b05-faad-44d5-b83e-4e9e341e8d2e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:54.872443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b35c4-b7a2-4c8a-9a61-1323873fc364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:54.882781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b35c4-b7a2-4c8a-9a61-1323873fc364 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:54.883306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b35c4-b7a2-4c8a-9a61-1323873fc364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:54.883806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b35c4-b7a2-4c8a-9a61-1323873fc364 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:54.885400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b35c4-b7a2-4c8a-9a61-1323873fc364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:54.889452 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b35c4-b7a2-4c8a-9a61-1323873fc364 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:54.964233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1001/1991] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:54 2022] POST /identity/v3/users => generated 292 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:54.968668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df477746-c29f-46dd-9364-c8ab72e488a5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:54.984558 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df477746-c29f-46dd-9364-c8ab72e488a5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:54.991575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3ffea-1b71-483e-b516-4862c213bfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:54.992514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3ffea-1b71-483e-b516-4862c213bfbd admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:54.992913 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3ffea-1b71-483e-b516-4862c213bfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:54.993333 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3ffea-1b71-483e-b516-4862c213bfbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:54.994312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3ffea-1b71-483e-b516-4862c213bfbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:54.996946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3ffea-1b71-483e-b516-4862c213bfbd admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:55.017273 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 991/1992] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:55.022179 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b35c4-b7a2-4c8a-9a61-1323873fc364 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:55.054616 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b35c4-b7a2-4c8a-9a61-1323873fc364 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:55.069582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1fa16-3d49-4c8b-99c1-04f4a4f29df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:55.071239 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1fa16-3d49-4c8b-99c1-04f4a4f29df8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:55.071730 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1fa16-3d49-4c8b-99c1-04f4a4f29df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:55.072414 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1fa16-3d49-4c8b-99c1-04f4a4f29df8 admin admin] PATH_INFO: `/v3/projects/fb4c3654dc4a43f38a37d75f99d56173/users/ed16dda203ac489f944507036d97e6f3/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:55.173803 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1fa16-3d49-4c8b-99c1-04f4a4f29df8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb4c3654dc4a43f38a37d75f99d56173, user_id=ed16dda203ac489f944507036d97e6f3, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:55.199683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1fa16-3d49-4c8b-99c1-04f4a4f29df8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:55.246427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:55.266325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1002/1993] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:55 2022] PUT /identity/v3/projects/fb4c3654dc4a43f38a37d75f99d56173/users/ed16dda203ac489f944507036d97e6f3/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:55.270983 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3ffea-1b71-483e-b516-4862c213bfbd admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:55.304364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3ffea-1b71-483e-b516-4862c213bfbd admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:55.311820 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1fa16-3d49-4c8b-99c1-04f4a4f29df8 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:55.315932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f7d1fa16-3d49-4c8b-99c1-04f4a4f29df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:55.483311 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52457a5-9393-48d5-bdb0-628de6d74f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:55.484019 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52457a5-9393-48d5-bdb0-628de6d74f75 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:55.484329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52457a5-9393-48d5-bdb0-628de6d74f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:55.484624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52457a5-9393-48d5-bdb0-628de6d74f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:55.485433 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52457a5-9393-48d5-bdb0-628de6d74f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:55.488682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52457a5-9393-48d5-bdb0-628de6d74f75 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:55.525680 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 992/1994] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:55.526725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52457a5-9393-48d5-bdb0-628de6d74f75 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:55.553560 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52457a5-9393-48d5-bdb0-628de6d74f75 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:55.563783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae88d18-aa16-4f97-9ff3-1172d61ef259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:55.564882 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae88d18-aa16-4f97-9ff3-1172d61ef259 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:55.565206 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae88d18-aa16-4f97-9ff3-1172d61ef259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:55.565558 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae88d18-aa16-4f97-9ff3-1172d61ef259 admin admin] PATH_INFO: `/v3/projects/fb4c3654dc4a43f38a37d75f99d56173/users/ed16dda203ac489f944507036d97e6f3/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:55.580099 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae88d18-aa16-4f97-9ff3-1172d61ef259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb4c3654dc4a43f38a37d75f99d56173, user_id=ed16dda203ac489f944507036d97e6f3, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:55.586881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae88d18-aa16-4f97-9ff3-1172d61ef259 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:55.625882 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 993/1995] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:55 2022] PUT /identity/v3/projects/fb4c3654dc4a43f38a37d75f99d56173/users/ed16dda203ac489f944507036d97e6f3/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:55.656881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1fa16-3d49-4c8b-99c1-04f4a4f29df8 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:55.759679 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440c90a-1c35-4b73-af19-6cca3ac9868a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:55.760264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440c90a-1c35-4b73-af19-6cca3ac9868a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:55.760529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440c90a-1c35-4b73-af19-6cca3ac9868a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:55.760802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440c90a-1c35-4b73-af19-6cca3ac9868a admin admin] PATH_INFO: `/v3/projects/6fdf399d28be49beaf81f071d2a04e30` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:55.762203 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440c90a-1c35-4b73-af19-6cca3ac9868a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fdf399d28be49beaf81f071d2a04e30)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:55.764219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440c90a-1c35-4b73-af19-6cca3ac9868a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:55.830630 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b440c90a-1c35-4b73-af19-6cca3ac9868a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fdf399d28be49beaf81f071d2a04e30', 'request_id': 'req-b440c90a-1c35-4b73-af19-6cca3ac9868a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:55.841316 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b440c90a-1c35-4b73-af19-6cca3ac9868a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fdf399d28be49beaf81f071d2a04e30', 'request_id': 'req-b440c90a-1c35-4b73-af19-6cca3ac9868a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:55.853193 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b440c90a-1c35-4b73-af19-6cca3ac9868a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fdf399d28be49beaf81f071d2a04e30', 'request_id': 'req-b440c90a-1c35-4b73-af19-6cca3ac9868a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:55.896868 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b440c90a-1c35-4b73-af19-6cca3ac9868a admin admin] The token cache is being invalidate because project 6fdf399d28be49beaf81f071d2a04e30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:31:55.898400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b440c90a-1c35-4b73-af19-6cca3ac9868a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:55.901270 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b440c90a-1c35-4b73-af19-6cca3ac9868a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:55.909798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1003/1996] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:31:55 2022] DELETE /identity/v3/projects/6fdf399d28be49beaf81f071d2a04e30 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:55.915545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d4d3b-a197-4846-9c5d-e0914d8222bd None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:55.915545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d4d3b-a197-4846-9c5d-e0914d8222bd None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:55.915987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d4d3b-a197-4846-9c5d-e0914d8222bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:55.942052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-937d4d3b-a197-4846-9c5d-e0914d8222bd None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:55.944818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-937d4d3b-a197-4846-9c5d-e0914d8222bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:56.038857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 994/1997] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:56.041031 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f3674-fae7-4ff4-aeaf-9e6f0a3092a3 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:56.041370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f3674-fae7-4ff4-aeaf-9e6f0a3092a3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:56.042097 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f3674-fae7-4ff4-aeaf-9e6f0a3092a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:56.079904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d94f3674-fae7-4ff4-aeaf-9e6f0a3092a3 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:56.080921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d94f3674-fae7-4ff4-aeaf-9e6f0a3092a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:56.115373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1004/1998] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:56.120909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d4d3b-a197-4846-9c5d-e0914d8222bd None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:56.138074 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d4d3b-a197-4846-9c5d-e0914d8222bd None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:56.145885 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3771bfc-204a-4d90-bd70-dfaea76fbfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:56.147501 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3771bfc-204a-4d90-bd70-dfaea76fbfad admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:56.147831 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3771bfc-204a-4d90-bd70-dfaea76fbfad admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:56.148140 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3771bfc-204a-4d90-bd70-dfaea76fbfad admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:56.148911 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3771bfc-204a-4d90-bd70-dfaea76fbfad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:31:56.149225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3771bfc-204a-4d90-bd70-dfaea76fbfad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:56.156298 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3771bfc-204a-4d90-bd70-dfaea76fbfad admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:56.179115 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 995/1999] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:31:56.184853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f3674-fae7-4ff4-aeaf-9e6f0a3092a3 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:56.200059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f3674-fae7-4ff4-aeaf-9e6f0a3092a3 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:56.225093 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed4155-c24a-4547-b73a-0dead5e162ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:56.227210 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed4155-c24a-4547-b73a-0dead5e162ac admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:56.227548 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed4155-c24a-4547-b73a-0dead5e162ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:56.227892 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed4155-c24a-4547-b73a-0dead5e162ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:56.229463 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed4155-c24a-4547-b73a-0dead5e162ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:56.234184 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed4155-c24a-4547-b73a-0dead5e162ac admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:56.287297 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1005/2000] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:56 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:56.291957 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3771bfc-204a-4d90-bd70-dfaea76fbfad admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:56.304585 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3771bfc-204a-4d90-bd70-dfaea76fbfad admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:56.389079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6443e72-d44b-4b13-88e7-82df534aee59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:56.389696 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6443e72-d44b-4b13-88e7-82df534aee59 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:56.389989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6443e72-d44b-4b13-88e7-82df534aee59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:56.390267 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6443e72-d44b-4b13-88e7-82df534aee59 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:56.391140 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6443e72-d44b-4b13-88e7-82df534aee59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:56.393523 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6443e72-d44b-4b13-88e7-82df534aee59 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:56.402354 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:56.437607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 996/2001] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:56 2022] POST /identity/v3/users => generated 291 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:56.441910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed4155-c24a-4547-b73a-0dead5e162ac admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:56.453464 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed4155-c24a-4547-b73a-0dead5e162ac admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:56.461674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e9893-b460-4e7e-a351-d822fd3d6109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:56.463860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e9893-b460-4e7e-a351-d822fd3d6109 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:56.464336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e9893-b460-4e7e-a351-d822fd3d6109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:56.464762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e9893-b460-4e7e-a351-d822fd3d6109 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:56.465751 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e9893-b460-4e7e-a351-d822fd3d6109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:56.467956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e9893-b460-4e7e-a351-d822fd3d6109 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:56.470244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:56.489745 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1006/2002] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:31:56.493794 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6443e72-d44b-4b13-88e7-82df534aee59 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:56.518768 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6443e72-d44b-4b13-88e7-82df534aee59 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:56.526961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bca8e6-d8e2-4ed9-851e-aa5d68c242eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:56.528573 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bca8e6-d8e2-4ed9-851e-aa5d68c242eb admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:56.529428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bca8e6-d8e2-4ed9-851e-aa5d68c242eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:56.530530 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bca8e6-d8e2-4ed9-851e-aa5d68c242eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:56.532446 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bca8e6-d8e2-4ed9-851e-aa5d68c242eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:56.537058 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bca8e6-d8e2-4ed9-851e-aa5d68c242eb admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:56.568696 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 997/2003] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:56.573119 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e9893-b460-4e7e-a351-d822fd3d6109 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:56.611213 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e9893-b460-4e7e-a351-d822fd3d6109 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:56.635673 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e41ee-72d3-4691-a60a-a7979d9d0566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:56.638749 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e41ee-72d3-4691-a60a-a7979d9d0566 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:56.642586 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e41ee-72d3-4691-a60a-a7979d9d0566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:56.643985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e41ee-72d3-4691-a60a-a7979d9d0566 admin admin] PATH_INFO: `/v3/projects/c4e2fb76a2f144bdbe4f76402b04346c/users/9c4cca4fb4194d078384cdb7a2734349/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:56.727621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e41ee-72d3-4691-a60a-a7979d9d0566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e2fb76a2f144bdbe4f76402b04346c, user_id=9c4cca4fb4194d078384cdb7a2734349, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:56.744912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e41ee-72d3-4691-a60a-a7979d9d0566 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:56.789445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1007/2004] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:56 2022] PUT /identity/v3/projects/c4e2fb76a2f144bdbe4f76402b04346c/users/9c4cca4fb4194d078384cdb7a2734349/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:56.802711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85532213-1c48-48e7-9880-6eb3f63ec92b None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:56.803086 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85532213-1c48-48e7-9880-6eb3f63ec92b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:56.803590 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85532213-1c48-48e7-9880-6eb3f63ec92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:56.850997 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-85532213-1c48-48e7-9880-6eb3f63ec92b None None] MFA Rules not processed for user `9c4cca4fb4194d078384cdb7a2734349`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:56.853886 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-85532213-1c48-48e7-9880-6eb3f63ec92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:57.073567 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e41ee-72d3-4691-a60a-a7979d9d0566 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:57.085450 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 998/2005] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:57.186618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e41ee-72d3-4691-a60a-a7979d9d0566 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:57.193270 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92830cf5-4340-4ecd-812a-40f72b176546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:57.195683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92830cf5-4340-4ecd-812a-40f72b176546 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:57.195683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92830cf5-4340-4ecd-812a-40f72b176546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:57.196033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92830cf5-4340-4ecd-812a-40f72b176546 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:57.196751 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92830cf5-4340-4ecd-812a-40f72b176546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:57.198885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92830cf5-4340-4ecd-812a-40f72b176546 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:57.240411 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1008/2006] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:31:57 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:57.242110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85532213-1c48-48e7-9880-6eb3f63ec92b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:57.259626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85532213-1c48-48e7-9880-6eb3f63ec92b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:57.403312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac324d05-8aeb-4e5a-bd8b-194c0fb4295f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:57.403312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac324d05-8aeb-4e5a-bd8b-194c0fb4295f admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:57.403312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac324d05-8aeb-4e5a-bd8b-194c0fb4295f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:57.403312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac324d05-8aeb-4e5a-bd8b-194c0fb4295f admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:57.403692 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac324d05-8aeb-4e5a-bd8b-194c0fb4295f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:57.409687 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac324d05-8aeb-4e5a-bd8b-194c0fb4295f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:57.476678 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 999/2007] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:31:57 2022] POST /identity/v3/users => generated 298 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:31:57.482697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92830cf5-4340-4ecd-812a-40f72b176546 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:57.542010 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92830cf5-4340-4ecd-812a-40f72b176546 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:57.562867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28ebfc-2d3a-4215-9756-aa9c10249b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:57.566207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28ebfc-2d3a-4215-9756-aa9c10249b4f admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:57.566566 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28ebfc-2d3a-4215-9756-aa9c10249b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:57.566857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28ebfc-2d3a-4215-9756-aa9c10249b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:57.568610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d28ebfc-2d3a-4215-9756-aa9c10249b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:57.570186 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d28ebfc-2d3a-4215-9756-aa9c10249b4f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:57.589547 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fa7f9-f6c3-4b73-b6f9-c5cc728b11b8 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:57.590563 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fa7f9-f6c3-4b73-b6f9-c5cc728b11b8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:57.591240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fa7f9-f6c3-4b73-b6f9-c5cc728b11b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:57.641251 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1009/2008] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:31:57.645604 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28ebfc-2d3a-4215-9756-aa9c10249b4f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:57.679702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28ebfc-2d3a-4215-9756-aa9c10249b4f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:57.697127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eed99d-e825-4bfc-8d1e-c5fc8f475330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:57.698604 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eed99d-e825-4bfc-8d1e-c5fc8f475330 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:57.699063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eed99d-e825-4bfc-8d1e-c5fc8f475330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:57.699345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eed99d-e825-4bfc-8d1e-c5fc8f475330 admin admin] PATH_INFO: `/v3/projects/293ade1e160443fd8177db487fe509fa/users/17618b1b44ba419a8ebee76e11504e31/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:57.705980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f95fa7f9-f6c3-4b73-b6f9-c5cc728b11b8 None None] MFA Rules not processed for user `ed16dda203ac489f944507036d97e6f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:57.715424 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f95fa7f9-f6c3-4b73-b6f9-c5cc728b11b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:57.735669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eed99d-e825-4bfc-8d1e-c5fc8f475330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=293ade1e160443fd8177db487fe509fa, user_id=17618b1b44ba419a8ebee76e11504e31, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:57.739682 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eed99d-e825-4bfc-8d1e-c5fc8f475330 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:57.773817 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1010/2009] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:57 2022] PUT /identity/v3/projects/293ade1e160443fd8177db487fe509fa/users/17618b1b44ba419a8ebee76e11504e31/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:57.775267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eed99d-e825-4bfc-8d1e-c5fc8f475330 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:57.915506 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:57.949475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eed99d-e825-4bfc-8d1e-c5fc8f475330 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:57.968451 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf420b66-9177-488a-a41b-66115a3c3e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:57.969532 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf420b66-9177-488a-a41b-66115a3c3e72 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:57.969819 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf420b66-9177-488a-a41b-66115a3c3e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:57.970105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf420b66-9177-488a-a41b-66115a3c3e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:57.971044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf420b66-9177-488a-a41b-66115a3c3e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:57.972726 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf420b66-9177-488a-a41b-66115a3c3e72 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:57.986953 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1011/2010] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:31:57.988254 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1000/2011] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:57 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:57.999289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf420b66-9177-488a-a41b-66115a3c3e72 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:58.005412 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fa7f9-f6c3-4b73-b6f9-c5cc728b11b8 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:58.006995 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f95fa7f9-f6c3-4b73-b6f9-c5cc728b11b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:58.016915 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf420b66-9177-488a-a41b-66115a3c3e72 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:58.025663 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009b981-f365-4d0a-afda-b1638f82fe13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:58.026932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009b981-f365-4d0a-afda-b1638f82fe13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:58.027312 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009b981-f365-4d0a-afda-b1638f82fe13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:58.027844 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009b981-f365-4d0a-afda-b1638f82fe13 admin admin] PATH_INFO: `/v3/projects/293ade1e160443fd8177db487fe509fa/users/17618b1b44ba419a8ebee76e11504e31/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:58.030976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009b981-f365-4d0a-afda-b1638f82fe13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=293ade1e160443fd8177db487fe509fa, user_id=17618b1b44ba419a8ebee76e11504e31, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:58.035694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009b981-f365-4d0a-afda-b1638f82fe13 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:58.065562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1012/2012] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:31:57 2022] PUT /identity/v3/projects/293ade1e160443fd8177db487fe509fa/users/17618b1b44ba419a8ebee76e11504e31/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:58.080544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07ad54-abbe-48f4-8989-0a4144f5094e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:58.081174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07ad54-abbe-48f4-8989-0a4144f5094e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:58.083418 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07ad54-abbe-48f4-8989-0a4144f5094e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:58.096910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:58.154048 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5f07ad54-abbe-48f4-8989-0a4144f5094e None None] MFA Rules not processed for user `17618b1b44ba419a8ebee76e11504e31`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:58.155627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5f07ad54-abbe-48f4-8989-0a4144f5094e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:58.157671 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fa7f9-f6c3-4b73-b6f9-c5cc728b11b8 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:58.283947 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1951f8-9ab3-4bff-8245-b09e5f1d2fc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:58.284627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1951f8-9ab3-4bff-8245-b09e5f1d2fc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:58.284915 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1951f8-9ab3-4bff-8245-b09e5f1d2fc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:58.285263 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1951f8-9ab3-4bff-8245-b09e5f1d2fc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:58.289919 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5f1951f8-9ab3-4bff-8245-b09e5f1d2fc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:58.308670 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1013/2013] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:31:58.316495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07ad54-abbe-48f4-8989-0a4144f5094e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:58.321931 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5f07ad54-abbe-48f4-8989-0a4144f5094e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:58.352149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07ad54-abbe-48f4-8989-0a4144f5094e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:58.357729 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ea067-c660-452c-9fc6-f043e58c49ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:58.358545 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ea067-c660-452c-9fc6-f043e58c49ff service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:58.358923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ea067-c660-452c-9fc6-f043e58c49ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:58.359284 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ea067-c660-452c-9fc6-f043e58c49ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:58.371790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ea067-c660-452c-9fc6-f043e58c49ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:58.373647 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ea067-c660-452c-9fc6-f043e58c49ff service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:58.386527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1014/2014] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:58.407869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1951f8-9ab3-4bff-8245-b09e5f1d2fc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:58.409441 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1951f8-9ab3-4bff-8245-b09e5f1d2fc5 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:58.421479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1001/2015] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:31:58.648133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ea067-c660-452c-9fc6-f043e58c49ff service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:58.678981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ea067-c660-452c-9fc6-f043e58c49ff service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:58.693269 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e59eb-4096-4ed5-a790-1b3b26fbe422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:58.696330 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e59eb-4096-4ed5-a790-1b3b26fbe422 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:58.696893 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e59eb-4096-4ed5-a790-1b3b26fbe422 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:58.697261 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e59eb-4096-4ed5-a790-1b3b26fbe422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:58.853632 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e59eb-4096-4ed5-a790-1b3b26fbe422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:58.861552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e59eb-4096-4ed5-a790-1b3b26fbe422 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:58.883575 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1015/2016] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:31:59.222025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1951f8-9ab3-4bff-8245-b09e5f1d2fc5 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:59.223755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5f1951f8-9ab3-4bff-8245-b09e5f1d2fc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:59.323875 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:59.359435 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1951f8-9ab3-4bff-8245-b09e5f1d2fc5 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:59.370639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e52877-8466-4398-9a65-21c81dc6f4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:59.373582 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e52877-8466-4398-9a65-21c81dc6f4d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:59.373582 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e52877-8466-4398-9a65-21c81dc6f4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:59.373582 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e52877-8466-4398-9a65-21c81dc6f4d7 admin admin] PATH_INFO: `/v3/users/99efebbb2a3e4f32b0a4ed87b1509c32` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:59.379125 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e52877-8466-4398-9a65-21c81dc6f4d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99efebbb2a3e4f32b0a4ed87b1509c32)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:59.385707 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e52877-8466-4398-9a65-21c81dc6f4d7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:59.497055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-30e52877-8466-4398-9a65-21c81dc6f4d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99efebbb2a3e4f32b0a4ed87b1509c32'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:59.530515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-30e52877-8466-4398-9a65-21c81dc6f4d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99efebbb2a3e4f32b0a4ed87b1509c32'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:59.547863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-30e52877-8466-4398-9a65-21c81dc6f4d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99efebbb2a3e4f32b0a4ed87b1509c32'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:59.549165 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-30e52877-8466-4398-9a65-21c81dc6f4d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99efebbb2a3e4f32b0a4ed87b1509c32'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:59.550420 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-30e52877-8466-4398-9a65-21c81dc6f4d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99efebbb2a3e4f32b0a4ed87b1509c32'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:59.575938 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1002/2017] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:59 2022] DELETE /identity/v3/users/99efebbb2a3e4f32b0a4ed87b1509c32 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:59.584255 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab9f92-dd4b-4487-b845-c3731a4a45e4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:59.585438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab9f92-dd4b-4487-b845-c3731a4a45e4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:59.585923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab9f92-dd4b-4487-b845-c3731a4a45e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:59.620318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3eab9f92-dd4b-4487-b845-c3731a4a45e4 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:59.622727 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3eab9f92-dd4b-4487-b845-c3731a4a45e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:59.644469 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e52877-8466-4398-9a65-21c81dc6f4d7 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:59.647156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-30e52877-8466-4398-9a65-21c81dc6f4d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:59.758946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1016/2018] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:31:59.770187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab9f92-dd4b-4487-b845-c3731a4a45e4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:31:59.772443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3eab9f92-dd4b-4487-b845-c3731a4a45e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:59.780858 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e52877-8466-4398-9a65-21c81dc6f4d7 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:59.787128 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f29f1-b6e0-44d8-9208-c41a86b9a35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:59.788461 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f29f1-b6e0-44d8-9208-c41a86b9a35f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:59.788764 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f29f1-b6e0-44d8-9208-c41a86b9a35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:59.789051 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f29f1-b6e0-44d8-9208-c41a86b9a35f admin admin] PATH_INFO: `/v3/users/9c4cca4fb4194d078384cdb7a2734349` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:59.790453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f29f1-b6e0-44d8-9208-c41a86b9a35f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c4cca4fb4194d078384cdb7a2734349)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:59.792493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f29f1-b6e0-44d8-9208-c41a86b9a35f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:59.809549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:59.858055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5b3f29f1-b6e0-44d8-9208-c41a86b9a35f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c4cca4fb4194d078384cdb7a2734349'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:59.883013 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5b3f29f1-b6e0-44d8-9208-c41a86b9a35f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c4cca4fb4194d078384cdb7a2734349'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:59.887034 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab9f92-dd4b-4487-b845-c3731a4a45e4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:31:59.893175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5b3f29f1-b6e0-44d8-9208-c41a86b9a35f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c4cca4fb4194d078384cdb7a2734349'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:59.893912 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5b3f29f1-b6e0-44d8-9208-c41a86b9a35f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c4cca4fb4194d078384cdb7a2734349'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:59.894109 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfb051-c8d1-4e3c-8a68-a8076884a747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:31:59.894724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5b3f29f1-b6e0-44d8-9208-c41a86b9a35f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c4cca4fb4194d078384cdb7a2734349'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:31:59.894905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfb051-c8d1-4e3c-8a68-a8076884a747 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:59.895094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfb051-c8d1-4e3c-8a68-a8076884a747 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:59.896201 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfb051-c8d1-4e3c-8a68-a8076884a747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:59.898238 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:31:59.898471 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-44cfb051-c8d1-4e3c-8a68-a8076884a747 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:31:59.913874 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfb051-c8d1-4e3c-8a68-a8076884a747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:31:59.915609 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfb051-c8d1-4e3c-8a68-a8076884a747 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:31:59.920263 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1003/2019] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:31:59 2022] DELETE /identity/v3/users/9c4cca4fb4194d078384cdb7a2734349 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:31:59.924950 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51194b-d75d-4978-8a02-a456ccc99b08 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:31:59.925227 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51194b-d75d-4978-8a02-a456ccc99b08 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:31:59.925579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51194b-d75d-4978-8a02-a456ccc99b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:31:59.928270 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1017/2020] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:31:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:31:59.953612 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3f51194b-d75d-4978-8a02-a456ccc99b08 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:31:59.955788 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3f51194b-d75d-4978-8a02-a456ccc99b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:00.057358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1004/2021] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:32:00.070044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfb051-c8d1-4e3c-8a68-a8076884a747 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:00.071877 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-44cfb051-c8d1-4e3c-8a68-a8076884a747 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:00.084564 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfb051-c8d1-4e3c-8a68-a8076884a747 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:00.196668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33344727-b0c0-4f43-84ec-d5e59df80c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:00.198005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33344727-b0c0-4f43-84ec-d5e59df80c46 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:00.198579 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33344727-b0c0-4f43-84ec-d5e59df80c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:00.199152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33344727-b0c0-4f43-84ec-d5e59df80c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:00.208774 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51194b-d75d-4978-8a02-a456ccc99b08 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:00.210525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3f51194b-d75d-4978-8a02-a456ccc99b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:00.219485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33344727-b0c0-4f43-84ec-d5e59df80c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:00.222407 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33344727-b0c0-4f43-84ec-d5e59df80c46 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:00.242811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1018/2022] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:32:00.291944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51194b-d75d-4978-8a02-a456ccc99b08 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:00.297054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e5725-7b6e-4518-b06b-d6029bf61d06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:00.297768 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e5725-7b6e-4518-b06b-d6029bf61d06 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:00.298171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e5725-7b6e-4518-b06b-d6029bf61d06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:00.298425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e5725-7b6e-4518-b06b-d6029bf61d06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:00.300372 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-249e5725-7b6e-4518-b06b-d6029bf61d06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:00.320596 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33344727-b0c0-4f43-84ec-d5e59df80c46 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:00.322236 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-33344727-b0c0-4f43-84ec-d5e59df80c46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:00.345429 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33344727-b0c0-4f43-84ec-d5e59df80c46 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:00.352597 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:00.354261 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:00.354803 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:00.355242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6 admin admin] PATH_INFO: `/v3/projects/e9270d02dc1f48eab107faf94f64c4dd` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:00.356709 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9270d02dc1f48eab107faf94f64c4dd)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:00.358976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:00.390502 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e5725-7b6e-4518-b06b-d6029bf61d06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:00.392054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e5725-7b6e-4518-b06b-d6029bf61d06 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:00.405115 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1005/2023] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:00 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:32:00.472326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9270d02dc1f48eab107faf94f64c4dd', 'request_id': 'req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:00.481697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9270d02dc1f48eab107faf94f64c4dd', 'request_id': 'req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:00.505761 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9270d02dc1f48eab107faf94f64c4dd', 'request_id': 'req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:00.555648 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6 admin admin] The token cache is being invalidate because project e9270d02dc1f48eab107faf94f64c4dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:00.556657 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:00.561490 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:00.565018 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1019/2024] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:00 2022] DELETE /identity/v3/projects/e9270d02dc1f48eab107faf94f64c4dd => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:00.570782 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e5725-7b6e-4518-b06b-d6029bf61d06 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:00.592813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e5725-7b6e-4518-b06b-d6029bf61d06 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:00.707400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:00.738043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46dad4-3cb8-454c-bcd4-d9cacafca738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:00.739029 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46dad4-3cb8-454c-bcd4-d9cacafca738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:00.739372 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46dad4-3cb8-454c-bcd4-d9cacafca738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:00.739794 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46dad4-3cb8-454c-bcd4-d9cacafca738 admin admin] PATH_INFO: `/v3/users/f9d7459ec76244e1a881998f9f42c064` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:00.741603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46dad4-3cb8-454c-bcd4-d9cacafca738 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9d7459ec76244e1a881998f9f42c064)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:00.743429 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46dad4-3cb8-454c-bcd4-d9cacafca738 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:00.822038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3c46dad4-3cb8-454c-bcd4-d9cacafca738 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9d7459ec76244e1a881998f9f42c064'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:00.860073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3c46dad4-3cb8-454c-bcd4-d9cacafca738 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9d7459ec76244e1a881998f9f42c064'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:00.873955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:00.873955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:00.897123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3c46dad4-3cb8-454c-bcd4-d9cacafca738 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9d7459ec76244e1a881998f9f42c064'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:00.898415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3c46dad4-3cb8-454c-bcd4-d9cacafca738 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9d7459ec76244e1a881998f9f42c064'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:00.899468 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3c46dad4-3cb8-454c-bcd4-d9cacafca738 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9d7459ec76244e1a881998f9f42c064'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:00.918208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd454d66-9fea-4eed-bbca-cba30c1b2bc6 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:00.929546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a405ce-9ba1-4900-b7d1-05d93aa0db5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:00.931500 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a405ce-9ba1-4900-b7d1-05d93aa0db5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:00.931500 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a405ce-9ba1-4900-b7d1-05d93aa0db5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:00.931842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a405ce-9ba1-4900-b7d1-05d93aa0db5b admin admin] PATH_INFO: `/v3/users/1e94688cd938492da8a03259f995fc17` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:00.935445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a405ce-9ba1-4900-b7d1-05d93aa0db5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e94688cd938492da8a03259f995fc17)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:00.937627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a405ce-9ba1-4900-b7d1-05d93aa0db5b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:00.947284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1006/2025] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:32:00 2022] DELETE /identity/v3/users/f9d7459ec76244e1a881998f9f42c064 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:00.952310 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46dad4-3cb8-454c-bcd4-d9cacafca738 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:00.954940 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3c46dad4-3cb8-454c-bcd4-d9cacafca738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:01.042288 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:01.058142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e2a405ce-9ba1-4900-b7d1-05d93aa0db5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e94688cd938492da8a03259f995fc17'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:01.060537 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e2a405ce-9ba1-4900-b7d1-05d93aa0db5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e94688cd938492da8a03259f995fc17'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:01.096557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e2a405ce-9ba1-4900-b7d1-05d93aa0db5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e94688cd938492da8a03259f995fc17'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:01.099209 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e2a405ce-9ba1-4900-b7d1-05d93aa0db5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e94688cd938492da8a03259f995fc17'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:01.112140 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46dad4-3cb8-454c-bcd4-d9cacafca738 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:01.121336 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205108b-0268-4d75-9e37-2d25b1d9f221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:01.121646 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e2a405ce-9ba1-4900-b7d1-05d93aa0db5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e94688cd938492da8a03259f995fc17'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:01.122948 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205108b-0268-4d75-9e37-2d25b1d9f221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:01.123540 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205108b-0268-4d75-9e37-2d25b1d9f221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:01.124145 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205108b-0268-4d75-9e37-2d25b1d9f221 admin admin] PATH_INFO: `/v3/projects/c4e2fb76a2f144bdbe4f76402b04346c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:01.126945 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a205108b-0268-4d75-9e37-2d25b1d9f221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4e2fb76a2f144bdbe4f76402b04346c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:01.131468 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a205108b-0268-4d75-9e37-2d25b1d9f221 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:01.150775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1020/2026] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:32:00 2022] DELETE /identity/v3/users/1e94688cd938492da8a03259f995fc17 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:01.156081 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38447efb-c643-49f2-8210-60078b4483c7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:01.156674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38447efb-c643-49f2-8210-60078b4483c7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:01.158008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38447efb-c643-49f2-8210-60078b4483c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:01.182238 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-38447efb-c643-49f2-8210-60078b4483c7 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:01.183895 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-38447efb-c643-49f2-8210-60078b4483c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:01.245828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a205108b-0268-4d75-9e37-2d25b1d9f221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4e2fb76a2f144bdbe4f76402b04346c', 'request_id': 'req-a205108b-0268-4d75-9e37-2d25b1d9f221'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:01.267320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a205108b-0268-4d75-9e37-2d25b1d9f221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4e2fb76a2f144bdbe4f76402b04346c', 'request_id': 'req-a205108b-0268-4d75-9e37-2d25b1d9f221'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:01.274262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:01.276845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a205108b-0268-4d75-9e37-2d25b1d9f221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4e2fb76a2f144bdbe4f76402b04346c', 'request_id': 'req-a205108b-0268-4d75-9e37-2d25b1d9f221'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:01.306921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a205108b-0268-4d75-9e37-2d25b1d9f221 admin admin] The token cache is being invalidate because project c4e2fb76a2f144bdbe4f76402b04346c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:01.308009 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a205108b-0268-4d75-9e37-2d25b1d9f221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:01.308756 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a205108b-0268-4d75-9e37-2d25b1d9f221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:01.311610 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1007/2027] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:00 2022] DELETE /identity/v3/projects/c4e2fb76a2f144bdbe4f76402b04346c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:01.319299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205108b-0268-4d75-9e37-2d25b1d9f221 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:01.324966 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a205108b-0268-4d75-9e37-2d25b1d9f221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:01.375239 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1021/2028] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:32:01.408419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38447efb-c643-49f2-8210-60078b4483c7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:01.412702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-38447efb-c643-49f2-8210-60078b4483c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:01.460881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205108b-0268-4d75-9e37-2d25b1d9f221 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:01.467951 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078fe63-ee56-46f3-a006-3f0fbd888874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:01.469948 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078fe63-ee56-46f3-a006-3f0fbd888874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:01.470431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078fe63-ee56-46f3-a006-3f0fbd888874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:01.470870 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078fe63-ee56-46f3-a006-3f0fbd888874 admin admin] PATH_INFO: `/v3/users/17618b1b44ba419a8ebee76e11504e31` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:01.473011 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078fe63-ee56-46f3-a006-3f0fbd888874 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17618b1b44ba419a8ebee76e11504e31)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:01.477302 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078fe63-ee56-46f3-a006-3f0fbd888874 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:01.554166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38447efb-c643-49f2-8210-60078b4483c7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:01.589255 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3078fe63-ee56-46f3-a006-3f0fbd888874 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17618b1b44ba419a8ebee76e11504e31'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:01.624434 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3078fe63-ee56-46f3-a006-3f0fbd888874 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17618b1b44ba419a8ebee76e11504e31'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:01.648239 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3078fe63-ee56-46f3-a006-3f0fbd888874 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17618b1b44ba419a8ebee76e11504e31'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:01.650307 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3078fe63-ee56-46f3-a006-3f0fbd888874 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17618b1b44ba419a8ebee76e11504e31'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:01.652991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3078fe63-ee56-46f3-a006-3f0fbd888874 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17618b1b44ba419a8ebee76e11504e31'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:01.676269 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec753f-916e-4977-b701-880c660e1879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:01.679703 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec753f-916e-4977-b701-880c660e1879 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:01.680169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec753f-916e-4977-b701-880c660e1879 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:01.680615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec753f-916e-4977-b701-880c660e1879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:01.687363 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-29ec753f-916e-4977-b701-880c660e1879 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:01.688886 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1008/2029] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:32:01 2022] DELETE /identity/v3/users/17618b1b44ba419a8ebee76e11504e31 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:01.730370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec753f-916e-4977-b701-880c660e1879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:01.735387 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec753f-916e-4977-b701-880c660e1879 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:01.762867 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078fe63-ee56-46f3-a006-3f0fbd888874 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:01.765505 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3078fe63-ee56-46f3-a006-3f0fbd888874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:01.801682 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1022/2030] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:32:01.988092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078fe63-ee56-46f3-a006-3f0fbd888874 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:01.998571 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fd092-d3fe-48c0-a181-3861f878acbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:02.000833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fd092-d3fe-48c0-a181-3861f878acbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:02.001484 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fd092-d3fe-48c0-a181-3861f878acbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:02.002438 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fd092-d3fe-48c0-a181-3861f878acbf admin admin] PATH_INFO: `/v3/projects/6f2ea80b14c74de7816681e9fc32a9fb` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:02.006458 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fd092-d3fe-48c0-a181-3861f878acbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f2ea80b14c74de7816681e9fc32a9fb)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:02.011182 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fd092-d3fe-48c0-a181-3861f878acbf admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:02.040228 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:02.208351 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a42fd092-d3fe-48c0-a181-3861f878acbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f2ea80b14c74de7816681e9fc32a9fb', 'request_id': 'req-a42fd092-d3fe-48c0-a181-3861f878acbf'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:02.223810 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a42fd092-d3fe-48c0-a181-3861f878acbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f2ea80b14c74de7816681e9fc32a9fb', 'request_id': 'req-a42fd092-d3fe-48c0-a181-3861f878acbf'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:02.233603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a42fd092-d3fe-48c0-a181-3861f878acbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f2ea80b14c74de7816681e9fc32a9fb', 'request_id': 'req-a42fd092-d3fe-48c0-a181-3861f878acbf'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:02.251265 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a42fd092-d3fe-48c0-a181-3861f878acbf admin admin] The token cache is being invalidate because project 6f2ea80b14c74de7816681e9fc32a9fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:02.251645 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a42fd092-d3fe-48c0-a181-3861f878acbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:02.252412 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a42fd092-d3fe-48c0-a181-3861f878acbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:02.258820 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1009/2031] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:01 2022] DELETE /identity/v3/projects/6f2ea80b14c74de7816681e9fc32a9fb => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:32:02.260477 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec753f-916e-4977-b701-880c660e1879 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:02.262792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-29ec753f-916e-4977-b701-880c660e1879 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:02.266250 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fd092-d3fe-48c0-a181-3861f878acbf admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:02.269006 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a42fd092-d3fe-48c0-a181-3861f878acbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:02.292607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec753f-916e-4977-b701-880c660e1879 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:02.315204 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:02.374645 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fd092-d3fe-48c0-a181-3861f878acbf admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:02.399638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934c83b-edf7-44b5-a76e-451fc0789be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:02.400797 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934c83b-edf7-44b5-a76e-451fc0789be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:02.401521 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934c83b-edf7-44b5-a76e-451fc0789be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:02.401744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934c83b-edf7-44b5-a76e-451fc0789be4 admin admin] PATH_INFO: `/v3/users/81b28dc8cd31434497e1264854dc104c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:02.403371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934c83b-edf7-44b5-a76e-451fc0789be4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81b28dc8cd31434497e1264854dc104c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:02.405687 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934c83b-edf7-44b5-a76e-451fc0789be4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:02.441721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a317a9-23ae-474f-a36a-ecb02429425a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:02.446359 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a317a9-23ae-474f-a36a-ecb02429425a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:02.447211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a317a9-23ae-474f-a36a-ecb02429425a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:02.452063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a317a9-23ae-474f-a36a-ecb02429425a admin admin] PATH_INFO: `/v3/users/184b6909ea6649f3b5eadceef3cca359` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:02.453549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a317a9-23ae-474f-a36a-ecb02429425a admin admin] RBAC: Authorizing `identity:delete_user(user_id=184b6909ea6649f3b5eadceef3cca359)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:02.460112 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a317a9-23ae-474f-a36a-ecb02429425a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:02.503877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6934c83b-edf7-44b5-a76e-451fc0789be4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81b28dc8cd31434497e1264854dc104c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:02.527097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6934c83b-edf7-44b5-a76e-451fc0789be4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81b28dc8cd31434497e1264854dc104c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:02.539591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6934c83b-edf7-44b5-a76e-451fc0789be4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81b28dc8cd31434497e1264854dc104c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:02.542092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6934c83b-edf7-44b5-a76e-451fc0789be4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81b28dc8cd31434497e1264854dc104c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:02.543639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6934c83b-edf7-44b5-a76e-451fc0789be4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81b28dc8cd31434497e1264854dc104c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:02.547464 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-17a317a9-23ae-474f-a36a-ecb02429425a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '184b6909ea6649f3b5eadceef3cca359'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:02.548181 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-17a317a9-23ae-474f-a36a-ecb02429425a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '184b6909ea6649f3b5eadceef3cca359'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:02.564916 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-17a317a9-23ae-474f-a36a-ecb02429425a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '184b6909ea6649f3b5eadceef3cca359'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:02.565802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-17a317a9-23ae-474f-a36a-ecb02429425a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '184b6909ea6649f3b5eadceef3cca359'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:02.566442 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1010/2032] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:32:02 2022] DELETE /identity/v3/users/81b28dc8cd31434497e1264854dc104c => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:02.587810 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-17a317a9-23ae-474f-a36a-ecb02429425a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '184b6909ea6649f3b5eadceef3cca359'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:02.603052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934c83b-edf7-44b5-a76e-451fc0789be4 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:02.605431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6934c83b-edf7-44b5-a76e-451fc0789be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:02.610493 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1023/2033] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:32:02 2022] DELETE /identity/v3/users/184b6909ea6649f3b5eadceef3cca359 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:02.638419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:02.754547 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934c83b-edf7-44b5-a76e-451fc0789be4 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:02.825588 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a317a9-23ae-474f-a36a-ecb02429425a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:02.828059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-17a317a9-23ae-474f-a36a-ecb02429425a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:02.878990 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7acfc0-4c26-469b-bbc4-24ead3f5db44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:02.881575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7acfc0-4c26-469b-bbc4-24ead3f5db44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:02.882267 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7acfc0-4c26-469b-bbc4-24ead3f5db44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:02.882871 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7acfc0-4c26-469b-bbc4-24ead3f5db44 admin admin] PATH_INFO: `/v3/projects/293ade1e160443fd8177db487fe509fa` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:02.886307 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7acfc0-4c26-469b-bbc4-24ead3f5db44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=293ade1e160443fd8177db487fe509fa)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:02.891075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7acfc0-4c26-469b-bbc4-24ead3f5db44 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:02.924046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a317a9-23ae-474f-a36a-ecb02429425a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:02.930097 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5e883-5680-4aa7-b55b-b44798fa1e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:02.931388 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5e883-5680-4aa7-b55b-b44798fa1e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:02.931723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5e883-5680-4aa7-b55b-b44798fa1e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:02.932090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5e883-5680-4aa7-b55b-b44798fa1e4d admin admin] PATH_INFO: `/v3/projects/f133e7b0810e431ba84313a672c5ace9` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:02.933801 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5e883-5680-4aa7-b55b-b44798fa1e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f133e7b0810e431ba84313a672c5ace9)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:02.935838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5e883-5680-4aa7-b55b-b44798fa1e4d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:02.990317 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bb7acfc0-4c26-469b-bbc4-24ead3f5db44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '293ade1e160443fd8177db487fe509fa', 'request_id': 'req-bb7acfc0-4c26-469b-bbc4-24ead3f5db44'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:02.995172 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:03.003436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bb7acfc0-4c26-469b-bbc4-24ead3f5db44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '293ade1e160443fd8177db487fe509fa', 'request_id': 'req-bb7acfc0-4c26-469b-bbc4-24ead3f5db44'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.013966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-37d5e883-5680-4aa7-b55b-b44798fa1e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f133e7b0810e431ba84313a672c5ace9', 'request_id': 'req-37d5e883-5680-4aa7-b55b-b44798fa1e4d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.015145 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bb7acfc0-4c26-469b-bbc4-24ead3f5db44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '293ade1e160443fd8177db487fe509fa', 'request_id': 'req-bb7acfc0-4c26-469b-bbc4-24ead3f5db44'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.027769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-37d5e883-5680-4aa7-b55b-b44798fa1e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f133e7b0810e431ba84313a672c5ace9', 'request_id': 'req-37d5e883-5680-4aa7-b55b-b44798fa1e4d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.039471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bb7acfc0-4c26-469b-bbc4-24ead3f5db44 admin admin] The token cache is being invalidate because project 293ade1e160443fd8177db487fe509fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:03.040161 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bb7acfc0-4c26-469b-bbc4-24ead3f5db44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.040952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bb7acfc0-4c26-469b-bbc4-24ead3f5db44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.041679 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-37d5e883-5680-4aa7-b55b-b44798fa1e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f133e7b0810e431ba84313a672c5ace9', 'request_id': 'req-37d5e883-5680-4aa7-b55b-b44798fa1e4d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.043386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1011/2034] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:02 2022] DELETE /identity/v3/projects/293ade1e160443fd8177db487fe509fa => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:32:03.049626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8921455-c700-427f-a761-f9fdc3502407 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:03.049926 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8921455-c700-427f-a761-f9fdc3502407 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:03.050237 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8921455-c700-427f-a761-f9fdc3502407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:03.062995 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-37d5e883-5680-4aa7-b55b-b44798fa1e4d admin admin] The token cache is being invalidate because project f133e7b0810e431ba84313a672c5ace9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:03.063577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-37d5e883-5680-4aa7-b55b-b44798fa1e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.064654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-37d5e883-5680-4aa7-b55b-b44798fa1e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.068529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1024/2035] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:02 2022] DELETE /identity/v3/projects/f133e7b0810e431ba84313a672c5ace9 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:03.074006 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a8921455-c700-427f-a761-f9fdc3502407 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:03.074267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5e883-5680-4aa7-b55b-b44798fa1e4d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:03.075291 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a8921455-c700-427f-a761-f9fdc3502407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:03.076083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-37d5e883-5680-4aa7-b55b-b44798fa1e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:03.186739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:03.206505 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5e883-5680-4aa7-b55b-b44798fa1e4d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:03.218170 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1012/2036] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:32:03.222920 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593f4e8-bd14-4e3f-801b-c2423ea8346b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:03.225431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593f4e8-bd14-4e3f-801b-c2423ea8346b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:03.226062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593f4e8-bd14-4e3f-801b-c2423ea8346b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:03.226758 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593f4e8-bd14-4e3f-801b-c2423ea8346b admin admin] PATH_INFO: `/v3/users/583cbec012e340328de15714f649bb13` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:03.230054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a949d1-544a-4417-a038-fd1b949d8354 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:03.230587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a949d1-544a-4417-a038-fd1b949d8354 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:03.231127 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a949d1-544a-4417-a038-fd1b949d8354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:03.231935 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593f4e8-bd14-4e3f-801b-c2423ea8346b admin admin] RBAC: Authorizing `identity:delete_user(user_id=583cbec012e340328de15714f649bb13)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:03.236690 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593f4e8-bd14-4e3f-801b-c2423ea8346b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:03.257515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-48a949d1-544a-4417-a038-fd1b949d8354 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:03.259575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-48a949d1-544a-4417-a038-fd1b949d8354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:03.274046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1013/2037] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:03.278321 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a949d1-544a-4417-a038-fd1b949d8354 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:03.281024 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-48a949d1-544a-4417-a038-fd1b949d8354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:03.296205 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a949d1-544a-4417-a038-fd1b949d8354 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:03.302954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4bb74-e78d-4c1c-b2a8-fe8eba4a1302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:03.304010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4bb74-e78d-4c1c-b2a8-fe8eba4a1302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:03.304436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4bb74-e78d-4c1c-b2a8-fe8eba4a1302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:03.304809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4bb74-e78d-4c1c-b2a8-fe8eba4a1302 admin admin] PATH_INFO: `/v3/projects/d2d7305d881a4ee29213497d3bab5937` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:03.306462 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4bb74-e78d-4c1c-b2a8-fe8eba4a1302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2d7305d881a4ee29213497d3bab5937)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:03.308668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4bb74-e78d-4c1c-b2a8-fe8eba4a1302 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:03.350897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c593f4e8-bd14-4e3f-801b-c2423ea8346b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '583cbec012e340328de15714f649bb13'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.351742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c593f4e8-bd14-4e3f-801b-c2423ea8346b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '583cbec012e340328de15714f649bb13'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.369688 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c593f4e8-bd14-4e3f-801b-c2423ea8346b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '583cbec012e340328de15714f649bb13'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.370502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c593f4e8-bd14-4e3f-801b-c2423ea8346b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '583cbec012e340328de15714f649bb13'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.382662 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c593f4e8-bd14-4e3f-801b-c2423ea8346b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '583cbec012e340328de15714f649bb13'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.409144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5ef4bb74-e78d-4c1c-b2a8-fe8eba4a1302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2d7305d881a4ee29213497d3bab5937', 'request_id': 'req-5ef4bb74-e78d-4c1c-b2a8-fe8eba4a1302'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.416189 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1025/2038] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:32:03 2022] DELETE /identity/v3/users/583cbec012e340328de15714f649bb13 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:03.416189 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593f4e8-bd14-4e3f-801b-c2423ea8346b admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:03.423294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5ef4bb74-e78d-4c1c-b2a8-fe8eba4a1302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2d7305d881a4ee29213497d3bab5937', 'request_id': 'req-5ef4bb74-e78d-4c1c-b2a8-fe8eba4a1302'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.424525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c593f4e8-bd14-4e3f-801b-c2423ea8346b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:03.432217 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5ef4bb74-e78d-4c1c-b2a8-fe8eba4a1302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2d7305d881a4ee29213497d3bab5937', 'request_id': 'req-5ef4bb74-e78d-4c1c-b2a8-fe8eba4a1302'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.466286 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5ef4bb74-e78d-4c1c-b2a8-fe8eba4a1302 admin admin] The token cache is being invalidate because project d2d7305d881a4ee29213497d3bab5937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:03.466584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5ef4bb74-e78d-4c1c-b2a8-fe8eba4a1302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.467338 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5ef4bb74-e78d-4c1c-b2a8-fe8eba4a1302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.470214 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1014/2039] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:03 2022] DELETE /identity/v3/projects/d2d7305d881a4ee29213497d3bab5937 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:03.477340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8a093-bcb9-45f5-8352-c40ade9c559d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:03.477621 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8a093-bcb9-45f5-8352-c40ade9c559d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:03.478036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8a093-bcb9-45f5-8352-c40ade9c559d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:03.518135 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-46c8a093-bcb9-45f5-8352-c40ade9c559d None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:03.522177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-46c8a093-bcb9-45f5-8352-c40ade9c559d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:03.566098 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:03.604016 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593f4e8-bd14-4e3f-801b-c2423ea8346b admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:03.610158 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ded4e-b857-4bcd-9b1d-e5d86bc7a1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:03.610902 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ded4e-b857-4bcd-9b1d-e5d86bc7a1bc admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:03.611197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ded4e-b857-4bcd-9b1d-e5d86bc7a1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:03.611498 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ded4e-b857-4bcd-9b1d-e5d86bc7a1bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:03.612386 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ded4e-b857-4bcd-9b1d-e5d86bc7a1bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:32:03.613252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ded4e-b857-4bcd-9b1d-e5d86bc7a1bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:03.617457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ded4e-b857-4bcd-9b1d-e5d86bc7a1bc admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:03.632469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1026/2040] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:32:03.637227 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ded4e-b857-4bcd-9b1d-e5d86bc7a1bc admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:03.639940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b90ded4e-b857-4bcd-9b1d-e5d86bc7a1bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:03.640884 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1015/2041] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:03.646766 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8a093-bcb9-45f5-8352-c40ade9c559d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:03.648628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-46c8a093-bcb9-45f5-8352-c40ade9c559d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:03.664572 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8a093-bcb9-45f5-8352-c40ade9c559d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:03.664849 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ded4e-b857-4bcd-9b1d-e5d86bc7a1bc admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:03.670031 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b0227-26d3-40a4-83fa-f33562254b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:03.670361 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc6fe0-d5e3-49ce-bae8-e1ec768be4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:03.670828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc6fe0-d5e3-49ce-bae8-e1ec768be4eb admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:03.671134 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc6fe0-d5e3-49ce-bae8-e1ec768be4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:03.671415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc6fe0-d5e3-49ce-bae8-e1ec768be4eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:03.671599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b0227-26d3-40a4-83fa-f33562254b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:03.671923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b0227-26d3-40a4-83fa-f33562254b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:03.672311 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b0227-26d3-40a4-83fa-f33562254b41 admin admin] PATH_INFO: `/v3/projects/c52a23fb983f4b28b5dae27ae10723a1` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:03.672464 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc6fe0-d5e3-49ce-bae8-e1ec768be4eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:03.673995 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b0227-26d3-40a4-83fa-f33562254b41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c52a23fb983f4b28b5dae27ae10723a1)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:03.674544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc6fe0-d5e3-49ce-bae8-e1ec768be4eb admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:03.676252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b0227-26d3-40a4-83fa-f33562254b41 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:03.729069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1016/2042] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:32:03 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:03.733931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb7a52-5060-41e8-bfe6-898bd506bd77 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:03.734808 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb7a52-5060-41e8-bfe6-898bd506bd77 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:03.738039 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb7a52-5060-41e8-bfe6-898bd506bd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:03.783246 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-367b0227-26d3-40a4-83fa-f33562254b41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c52a23fb983f4b28b5dae27ae10723a1', 'request_id': 'req-367b0227-26d3-40a4-83fa-f33562254b41'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.787302 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-abbb7a52-5060-41e8-bfe6-898bd506bd77 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:03.788758 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-abbb7a52-5060-41e8-bfe6-898bd506bd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:03.792375 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-367b0227-26d3-40a4-83fa-f33562254b41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c52a23fb983f4b28b5dae27ae10723a1', 'request_id': 'req-367b0227-26d3-40a4-83fa-f33562254b41'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.805795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-367b0227-26d3-40a4-83fa-f33562254b41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c52a23fb983f4b28b5dae27ae10723a1', 'request_id': 'req-367b0227-26d3-40a4-83fa-f33562254b41'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.846234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-367b0227-26d3-40a4-83fa-f33562254b41 admin admin] The token cache is being invalidate because project c52a23fb983f4b28b5dae27ae10723a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:03.846532 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-367b0227-26d3-40a4-83fa-f33562254b41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.847354 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-367b0227-26d3-40a4-83fa-f33562254b41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:03.854177 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1027/2043] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:03 2022] DELETE /identity/v3/projects/c52a23fb983f4b28b5dae27ae10723a1 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:03.854177 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b0227-26d3-40a4-83fa-f33562254b41 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:03.856249 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-367b0227-26d3-40a4-83fa-f33562254b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:03.956306 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:03.991505 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1017/2044] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:03.995063 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b9ac8-b95e-4714-b8d5-1edd7c2f01d4 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:03.995063 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b9ac8-b95e-4714-b8d5-1edd7c2f01d4 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:03.995063 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b9ac8-b95e-4714-b8d5-1edd7c2f01d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.002687 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b0227-26d3-40a4-83fa-f33562254b41 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:04.010044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854555f3-8b80-4d7f-8f09-8abab71de82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:04.011788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854555f3-8b80-4d7f-8f09-8abab71de82e admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.012263 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854555f3-8b80-4d7f-8f09-8abab71de82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.012739 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854555f3-8b80-4d7f-8f09-8abab71de82e admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.013655 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854555f3-8b80-4d7f-8f09-8abab71de82e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:04.015433 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-327b9ac8-b95e-4714-b8d5-1edd7c2f01d4 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:04.017378 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-327b9ac8-b95e-4714-b8d5-1edd7c2f01d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:04.018427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854555f3-8b80-4d7f-8f09-8abab71de82e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:04.034972 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1018/2045] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:04.036368 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b9ac8-b95e-4714-b8d5-1edd7c2f01d4 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:04.048782 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-327b9ac8-b95e-4714-b8d5-1edd7c2f01d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:04.070768 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1028/2046] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:32:03 2022] POST /identity/v3/users => generated 287 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:04.074497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854555f3-8b80-4d7f-8f09-8abab71de82e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:04.085859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b9ac8-b95e-4714-b8d5-1edd7c2f01d4 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:04.095188 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854555f3-8b80-4d7f-8f09-8abab71de82e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:04.096484 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:04.097711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.099068 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.099540 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7 admin admin] PATH_INFO: `/v3/projects/5ad49bbe963c4ac5b317ef7ea7d5452c` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.101811 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ad49bbe963c4ac5b317ef7ea7d5452c)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:04.104328 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:04.104993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f702f2f-0502-408c-8231-1977e84f75a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:04.106423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f702f2f-0502-408c-8231-1977e84f75a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.107100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f702f2f-0502-408c-8231-1977e84f75a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.107693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f702f2f-0502-408c-8231-1977e84f75a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.109638 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f702f2f-0502-408c-8231-1977e84f75a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:32:04.110242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f702f2f-0502-408c-8231-1977e84f75a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:04.113106 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f702f2f-0502-408c-8231-1977e84f75a2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:04.131495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1029/2047] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:32:04.132634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ce4ae-f7f8-40f6-bc83-66c28f96eddd None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.132956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ce4ae-f7f8-40f6-bc83-66c28f96eddd None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.133241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ce4ae-f7f8-40f6-bc83-66c28f96eddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.157548 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-108ce4ae-f7f8-40f6-bc83-66c28f96eddd None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:04.159022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-108ce4ae-f7f8-40f6-bc83-66c28f96eddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:04.169759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1030/2048] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:04.170614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ce4ae-f7f8-40f6-bc83-66c28f96eddd None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:04.188091 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ce4ae-f7f8-40f6-bc83-66c28f96eddd None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:04.198769 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ad49bbe963c4ac5b317ef7ea7d5452c', 'request_id': 'req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:04.210815 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ad49bbe963c4ac5b317ef7ea7d5452c', 'request_id': 'req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:04.222164 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ad49bbe963c4ac5b317ef7ea7d5452c', 'request_id': 'req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:04.234422 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:04.247996 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7 admin admin] The token cache is being invalidate because project 5ad49bbe963c4ac5b317ef7ea7d5452c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:04.248528 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:04.249782 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:04.252735 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1019/2049] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:04 2022] DELETE /identity/v3/projects/5ad49bbe963c4ac5b317ef7ea7d5452c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:04.254247 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:04.258585 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:04.276100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51936b4-7dbf-40c6-8d1e-59b65253ee3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:04.277743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51936b4-7dbf-40c6-8d1e-59b65253ee3f admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.278204 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51936b4-7dbf-40c6-8d1e-59b65253ee3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.278667 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51936b4-7dbf-40c6-8d1e-59b65253ee3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.280792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51936b4-7dbf-40c6-8d1e-59b65253ee3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:04.284082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51936b4-7dbf-40c6-8d1e-59b65253ee3f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:04.305681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1031/2050] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:04.305681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51936b4-7dbf-40c6-8d1e-59b65253ee3f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:04.308027 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b51936b4-7dbf-40c6-8d1e-59b65253ee3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:04.331318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51936b4-7dbf-40c6-8d1e-59b65253ee3f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:04.340509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fbd47-1413-4993-ab4e-734f40a330a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:04.341600 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fbd47-1413-4993-ab4e-734f40a330a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.342060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fbd47-1413-4993-ab4e-734f40a330a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.342519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fbd47-1413-4993-ab4e-734f40a330a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.343873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fbd47-1413-4993-ab4e-734f40a330a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:32:04.344333 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fbd47-1413-4993-ab4e-734f40a330a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:04.347095 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fbd47-1413-4993-ab4e-734f40a330a5 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:04.367833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1032/2051] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:32:04.370062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff61067-b1a6-4662-9b5b-8b31827de548 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.370513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff61067-b1a6-4662-9b5b-8b31827de548 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.370965 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff61067-b1a6-4662-9b5b-8b31827de548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.381002 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed455e2-9b49-40f2-9551-fbcbd5e392b7 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:04.388023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2aed73-8d82-4906-aa13-0d7a058bfebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:04.389082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2aed73-8d82-4906-aa13-0d7a058bfebe admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.389588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2aed73-8d82-4906-aa13-0d7a058bfebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.390041 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2aed73-8d82-4906-aa13-0d7a058bfebe admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.391210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2aed73-8d82-4906-aa13-0d7a058bfebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:04.394293 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2aed73-8d82-4906-aa13-0d7a058bfebe admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:04.400537 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-6ff61067-b1a6-4662-9b5b-8b31827de548 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:04.401809 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6ff61067-b1a6-4662-9b5b-8b31827de548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:04.411529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1033/2052] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:04.412620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff61067-b1a6-4662-9b5b-8b31827de548 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:04.414131 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6ff61067-b1a6-4662-9b5b-8b31827de548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:04.428198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1020/2053] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:32:04 2022] POST /identity/v3/projects => generated 367 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:04.429422 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff61067-b1a6-4662-9b5b-8b31827de548 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:04.430009 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2aed73-8d82-4906-aa13-0d7a058bfebe admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:04.508107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:04.508623 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a0056-43e4-4b0c-8870-bb8981c00a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:04.510165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a0056-43e4-4b0c-8870-bb8981c00a89 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.510786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a0056-43e4-4b0c-8870-bb8981c00a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.511283 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a0056-43e4-4b0c-8870-bb8981c00a89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.512717 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a0056-43e4-4b0c-8870-bb8981c00a89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:04.515961 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a0056-43e4-4b0c-8870-bb8981c00a89 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:04.516366 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2aed73-8d82-4906-aa13-0d7a058bfebe admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:04.526138 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab3eff-b9df-44f0-a1b5-8889d69c2168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:04.526931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab3eff-b9df-44f0-a1b5-8889d69c2168 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.527352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab3eff-b9df-44f0-a1b5-8889d69c2168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.527848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab3eff-b9df-44f0-a1b5-8889d69c2168 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.528640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab3eff-b9df-44f0-a1b5-8889d69c2168 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:04.533743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1034/2054] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:04.533743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61c096-9696-4c27-82c8-907d8eaf14ca None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.533743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61c096-9696-4c27-82c8-907d8eaf14ca None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.533743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61c096-9696-4c27-82c8-907d8eaf14ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.534345 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab3eff-b9df-44f0-a1b5-8889d69c2168 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:04.552780 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-4d61c096-9696-4c27-82c8-907d8eaf14ca None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:04.554133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4d61c096-9696-4c27-82c8-907d8eaf14ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:04.561617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1021/2055] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:32:04 2022] POST /identity/v3/projects => generated 369 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:04.562444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab3eff-b9df-44f0-a1b5-8889d69c2168 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:04.569176 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1035/2056] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:04.569176 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61c096-9696-4c27-82c8-907d8eaf14ca None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:04.577894 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61c096-9696-4c27-82c8-907d8eaf14ca None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:04.649356 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033aa58-2079-438b-8954-8c2f1a842ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:04.650081 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab3eff-b9df-44f0-a1b5-8889d69c2168 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:04.650434 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033aa58-2079-438b-8954-8c2f1a842ab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.651131 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033aa58-2079-438b-8954-8c2f1a842ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.651426 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033aa58-2079-438b-8954-8c2f1a842ab8 admin admin] PATH_INFO: `/v3/projects/63e787de17d9440faf96a0df0649bf2a/users/5f5f78cec93d4b499357685f7e20536e/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.655367 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89380418-240c-49f7-be66-200f0be22e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:04.656062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89380418-240c-49f7-be66-200f0be22e04 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.656382 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89380418-240c-49f7-be66-200f0be22e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.656630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89380418-240c-49f7-be66-200f0be22e04 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.658310 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89380418-240c-49f7-be66-200f0be22e04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:04.659358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89380418-240c-49f7-be66-200f0be22e04 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:04.675048 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033aa58-2079-438b-8954-8c2f1a842ab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e787de17d9440faf96a0df0649bf2a, user_id=5f5f78cec93d4b499357685f7e20536e, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:04.678503 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033aa58-2079-438b-8954-8c2f1a842ab8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:04.690435 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1022/2057] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:32:04 2022] POST /identity/v3/users => generated 284 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:04.690435 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89380418-240c-49f7-be66-200f0be22e04 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:04.697066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1036/2058] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:32:04 2022] PUT /identity/v3/projects/63e787de17d9440faf96a0df0649bf2a/users/5f5f78cec93d4b499357685f7e20536e/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:04.698037 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033aa58-2079-438b-8954-8c2f1a842ab8 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:04.702507 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89380418-240c-49f7-be66-200f0be22e04 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:04.723451 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:04.774244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033aa58-2079-438b-8954-8c2f1a842ab8 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:04.780088 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4719617-708f-4ef9-9409-7f0680807c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:04.780793 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4719617-708f-4ef9-9409-7f0680807c4f admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.781077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4719617-708f-4ef9-9409-7f0680807c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.781397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4719617-708f-4ef9-9409-7f0680807c4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.782154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4719617-708f-4ef9-9409-7f0680807c4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:32:04.782470 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4719617-708f-4ef9-9409-7f0680807c4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:04.784121 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4719617-708f-4ef9-9409-7f0680807c4f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:04.797512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1037/2059] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:32:04.798393 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4719617-708f-4ef9-9409-7f0680807c4f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:04.798964 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d324a6-6774-43ed-a363-889e99c118ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:04.799925 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d324a6-6774-43ed-a363-889e99c118ca admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.800424 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d324a6-6774-43ed-a363-889e99c118ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.800932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d324a6-6774-43ed-a363-889e99c118ca admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.802444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d324a6-6774-43ed-a363-889e99c118ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:04.805169 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d324a6-6774-43ed-a363-889e99c118ca admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:04.811829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4719617-708f-4ef9-9409-7f0680807c4f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:04.820026 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e68746-e7e6-43ff-bab8-f6200402c0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:04.821448 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e68746-e7e6-43ff-bab8-f6200402c0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.822003 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e68746-e7e6-43ff-bab8-f6200402c0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.822497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e68746-e7e6-43ff-bab8-f6200402c0e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.823532 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e68746-e7e6-43ff-bab8-f6200402c0e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:04.825702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e68746-e7e6-43ff-bab8-f6200402c0e9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:04.830048 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1023/2060] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:32:04 2022] POST /identity/v3/users => generated 285 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:04.831411 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b6f2c-2103-46de-9147-2e4d4b2756f0 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.831705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b6f2c-2103-46de-9147-2e4d4b2756f0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.832000 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b6f2c-2103-46de-9147-2e4d4b2756f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.843913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1038/2061] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:04.845680 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e68746-e7e6-43ff-bab8-f6200402c0e9 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:04.857528 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e68746-e7e6-43ff-bab8-f6200402c0e9 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:04.863046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41265d96-54f3-4dfc-b171-31cdceaf4567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:04.864244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41265d96-54f3-4dfc-b171-31cdceaf4567 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.864522 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41265d96-54f3-4dfc-b171-31cdceaf4567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.864864 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41265d96-54f3-4dfc-b171-31cdceaf4567 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.865551 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41265d96-54f3-4dfc-b171-31cdceaf4567 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:04.865806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-fb2b6f2c-2103-46de-9147-2e4d4b2756f0 None None] MFA Rules not processed for user `5f5f78cec93d4b499357685f7e20536e`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:04.867088 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fb2b6f2c-2103-46de-9147-2e4d4b2756f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:04.867495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41265d96-54f3-4dfc-b171-31cdceaf4567 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:04.897856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1039/2062] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:32:04 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:04.898791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41265d96-54f3-4dfc-b171-31cdceaf4567 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:04.924971 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:04.956562 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1024/2063] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:04.963788 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b6f2c-2103-46de-9147-2e4d4b2756f0 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:04.972707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41265d96-54f3-4dfc-b171-31cdceaf4567 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:04.975575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b6f2c-2103-46de-9147-2e4d4b2756f0 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:04.978975 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f543086-33f9-4d45-93aa-2ee2f68d7e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:04.979687 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f543086-33f9-4d45-93aa-2ee2f68d7e5f admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.979972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f543086-33f9-4d45-93aa-2ee2f68d7e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.980244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f543086-33f9-4d45-93aa-2ee2f68d7e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.980868 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20839d00-fc16-42a4-94aa-694312fb67d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:04.980997 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f543086-33f9-4d45-93aa-2ee2f68d7e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:04.981596 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20839d00-fc16-42a4-94aa-694312fb67d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:04.981881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20839d00-fc16-42a4-94aa-694312fb67d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:04.982160 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20839d00-fc16-42a4-94aa-694312fb67d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:04.982653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f543086-33f9-4d45-93aa-2ee2f68d7e5f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:04.982895 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20839d00-fc16-42a4-94aa-694312fb67d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:04.984612 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20839d00-fc16-42a4-94aa-694312fb67d6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:04.999582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1040/2064] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:04.999582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f543086-33f9-4d45-93aa-2ee2f68d7e5f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:04.999987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1025/2065] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:04.999987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20839d00-fc16-42a4-94aa-694312fb67d6 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:04.999987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-20839d00-fc16-42a4-94aa-694312fb67d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:05.013314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f543086-33f9-4d45-93aa-2ee2f68d7e5f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:05.018998 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1dbd4-d8b1-4a84-85f2-80e4f413eec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:05.019693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1dbd4-d8b1-4a84-85f2-80e4f413eec4 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:05.020012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1dbd4-d8b1-4a84-85f2-80e4f413eec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:05.020811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1dbd4-d8b1-4a84-85f2-80e4f413eec4 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:05.022116 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1dbd4-d8b1-4a84-85f2-80e4f413eec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:05.024949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1dbd4-d8b1-4a84-85f2-80e4f413eec4 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:05.055952 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1041/2066] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:32:04 2022] POST /identity/v3/users => generated 283 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:05.056868 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1dbd4-d8b1-4a84-85f2-80e4f413eec4 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:05.071454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1dbd4-d8b1-4a84-85f2-80e4f413eec4 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:05.076848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7c4bc-4ff7-4346-b582-e7743c76bdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:05.077564 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7c4bc-4ff7-4346-b582-e7743c76bdd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:05.077854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7c4bc-4ff7-4346-b582-e7743c76bdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:05.078134 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7c4bc-4ff7-4346-b582-e7743c76bdd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:05.078884 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed7c4bc-4ff7-4346-b582-e7743c76bdd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:05.080550 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed7c4bc-4ff7-4346-b582-e7743c76bdd7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:05.087720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20839d00-fc16-42a4-94aa-694312fb67d6 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:05.092084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2dc67d-73a5-4fe6-8d23-b7d9ea575c52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:05.092792 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dc67d-73a5-4fe6-8d23-b7d9ea575c52 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:05.093092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dc67d-73a5-4fe6-8d23-b7d9ea575c52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:05.093536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dc67d-73a5-4fe6-8d23-b7d9ea575c52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:05.097353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1042/2067] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:05.097353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7c4bc-4ff7-4346-b582-e7743c76bdd7 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:05.115779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7c4bc-4ff7-4346-b582-e7743c76bdd7 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:05.123250 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83112f27-3c04-4ab3-b68b-614def6eb584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:05.126570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83112f27-3c04-4ab3-b68b-614def6eb584 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:05.127025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83112f27-3c04-4ab3-b68b-614def6eb584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:05.128295 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83112f27-3c04-4ab3-b68b-614def6eb584 admin admin] PATH_INFO: `/v3/projects/b34afc12461b4acc9ea58364103e0710/users/91578702a34b4a1d89127c701cfe14eb/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:05.155122 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83112f27-3c04-4ab3-b68b-614def6eb584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34afc12461b4acc9ea58364103e0710, user_id=91578702a34b4a1d89127c701cfe14eb, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:05.158545 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83112f27-3c04-4ab3-b68b-614def6eb584 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:05.180829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1043/2068] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:32:05 2022] PUT /identity/v3/projects/b34afc12461b4acc9ea58364103e0710/users/91578702a34b4a1d89127c701cfe14eb/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:05.181747 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83112f27-3c04-4ab3-b68b-614def6eb584 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:05.206309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2dc67d-73a5-4fe6-8d23-b7d9ea575c52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:05.208920 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2dc67d-73a5-4fe6-8d23-b7d9ea575c52 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:05.220345 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1026/2069] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:32:05.221202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2dc67d-73a5-4fe6-8d23-b7d9ea575c52 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:05.236030 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2dc67d-73a5-4fe6-8d23-b7d9ea575c52 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:05.258028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:05.329700 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa3914-5645-4a39-b78d-d663e3662e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:05.334783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa3914-5645-4a39-b78d-d663e3662e20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:05.335757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa3914-5645-4a39-b78d-d663e3662e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:05.336936 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa3914-5645-4a39-b78d-d663e3662e20 admin admin] PATH_INFO: `/v3/projects/6e31c7d7676348a3aae0f5060ad557bc/users/d80b4a937e454b98b4a4931d89acac94/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:05.346229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83112f27-3c04-4ab3-b68b-614def6eb584 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:05.355924 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6add2-16ce-4595-8a3d-ddcaa229a423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:05.357066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6add2-16ce-4595-8a3d-ddcaa229a423 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:05.357613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6add2-16ce-4595-8a3d-ddcaa229a423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:05.358119 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6add2-16ce-4595-8a3d-ddcaa229a423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:05.359699 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6add2-16ce-4595-8a3d-ddcaa229a423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:05.363019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6add2-16ce-4595-8a3d-ddcaa229a423 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:05.373839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa3914-5645-4a39-b78d-d663e3662e20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e31c7d7676348a3aae0f5060ad557bc, user_id=d80b4a937e454b98b4a4931d89acac94, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:05.380242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa3914-5645-4a39-b78d-d663e3662e20 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:05.384484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1044/2070] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:05.387133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1b949-bcc2-4748-85e8-5ee62dfef1ce None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:05.387481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1b949-bcc2-4748-85e8-5ee62dfef1ce None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:05.387881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1b949-bcc2-4748-85e8-5ee62dfef1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:05.407120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1027/2071] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:32:05 2022] PUT /identity/v3/projects/6e31c7d7676348a3aae0f5060ad557bc/users/d80b4a937e454b98b4a4931d89acac94/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:05.412203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa3914-5645-4a39-b78d-d663e3662e20 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:05.430846 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-eeb1b949-bcc2-4748-85e8-5ee62dfef1ce None None] MFA Rules not processed for user `91578702a34b4a1d89127c701cfe14eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:05.433427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-eeb1b949-bcc2-4748-85e8-5ee62dfef1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:05.470310 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:05.562041 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa3914-5645-4a39-b78d-d663e3662e20 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:05.572755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a2d26-99d0-4670-903f-0dcb3b262be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:05.574361 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a2d26-99d0-4670-903f-0dcb3b262be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:05.574993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a2d26-99d0-4670-903f-0dcb3b262be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:05.575507 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a2d26-99d0-4670-903f-0dcb3b262be5 admin admin] PATH_INFO: `/v3/users/5f5f78cec93d4b499357685f7e20536e` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:05.578075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a2d26-99d0-4670-903f-0dcb3b262be5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f5f78cec93d4b499357685f7e20536e)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:05.581455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a2d26-99d0-4670-903f-0dcb3b262be5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:05.606387 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1045/2072] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:05 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:05.607050 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1b949-bcc2-4748-85e8-5ee62dfef1ce None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:05.623699 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1b949-bcc2-4748-85e8-5ee62dfef1ce None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:05.630669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba99f19-ec58-4210-adbe-634552d2bd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:05.632026 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba99f19-ec58-4210-adbe-634552d2bd5b admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:05.632988 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba99f19-ec58-4210-adbe-634552d2bd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:05.633585 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba99f19-ec58-4210-adbe-634552d2bd5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:05.635054 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba99f19-ec58-4210-adbe-634552d2bd5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:05.639594 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba99f19-ec58-4210-adbe-634552d2bd5b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:05.662988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7a3a2d26-99d0-4670-903f-0dcb3b262be5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f5f78cec93d4b499357685f7e20536e'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:05.667674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1046/2073] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:05.667674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b4953-e6a3-460e-b1e8-e2ebf05b614f None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:05.667674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b4953-e6a3-460e-b1e8-e2ebf05b614f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:05.668001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b4953-e6a3-460e-b1e8-e2ebf05b614f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:05.696289 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7a3a2d26-99d0-4670-903f-0dcb3b262be5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f5f78cec93d4b499357685f7e20536e'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:05.709637 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7a3a2d26-99d0-4670-903f-0dcb3b262be5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f5f78cec93d4b499357685f7e20536e'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:05.710551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7a3a2d26-99d0-4670-903f-0dcb3b262be5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f5f78cec93d4b499357685f7e20536e'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:05.711693 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7a3a2d26-99d0-4670-903f-0dcb3b262be5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f5f78cec93d4b499357685f7e20536e'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:05.712005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-257b4953-e6a3-460e-b1e8-e2ebf05b614f None None] MFA Rules not processed for user `d80b4a937e454b98b4a4931d89acac94`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:05.713457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-257b4953-e6a3-460e-b1e8-e2ebf05b614f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:05.731280 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1028/2074] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:32:05 2022] DELETE /identity/v3/users/5f5f78cec93d4b499357685f7e20536e => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:05.732538 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a2d26-99d0-4670-903f-0dcb3b262be5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:05.734622 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7a3a2d26-99d0-4670-903f-0dcb3b262be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:05.811250 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a2d26-99d0-4670-903f-0dcb3b262be5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:05.814410 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1047/2075] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:05.816064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b4953-e6a3-460e-b1e8-e2ebf05b614f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:05.819341 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-257b4953-e6a3-460e-b1e8-e2ebf05b614f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:05.823302 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a055216-b14c-47f1-a42e-31c9477d3e6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:05.824022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a055216-b14c-47f1-a42e-31c9477d3e6d service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:05.824301 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a055216-b14c-47f1-a42e-31c9477d3e6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:05.824718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a055216-b14c-47f1-a42e-31c9477d3e6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:05.826767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3a055216-b14c-47f1-a42e-31c9477d3e6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:05.838099 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b4953-e6a3-460e-b1e8-e2ebf05b614f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:05.874207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:05.905808 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:05.923281 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1475474-20cb-4a1d-a1f8-a18b16f25423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:05.925030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1475474-20cb-4a1d-a1f8-a18b16f25423 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:05.925508 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1475474-20cb-4a1d-a1f8-a18b16f25423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:05.926043 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1475474-20cb-4a1d-a1f8-a18b16f25423 admin admin] PATH_INFO: `/v3/projects/2ce970851e074039b4c3b010bcaa7b06/users/3387b03fa72c4e4ab5a2d633ae6a5989/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:05.926175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a055216-b14c-47f1-a42e-31c9477d3e6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:05.928127 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a055216-b14c-47f1-a42e-31c9477d3e6d service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:05.938764 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1029/2076] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:05 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:32:05.940223 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251df89-e02b-4096-b83b-e018955f9fb6 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:05.940525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251df89-e02b-4096-b83b-e018955f9fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:05.940812 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251df89-e02b-4096-b83b-e018955f9fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:05.959465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1475474-20cb-4a1d-a1f8-a18b16f25423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce970851e074039b4c3b010bcaa7b06, user_id=3387b03fa72c4e4ab5a2d633ae6a5989, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:05.962660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1475474-20cb-4a1d-a1f8-a18b16f25423 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:05.969705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-8251df89-e02b-4096-b83b-e018955f9fb6 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:05.971178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8251df89-e02b-4096-b83b-e018955f9fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:05.980660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1048/2077] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:32:05 2022] PUT /identity/v3/projects/2ce970851e074039b4c3b010bcaa7b06/users/3387b03fa72c4e4ab5a2d633ae6a5989/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:05.981660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1475474-20cb-4a1d-a1f8-a18b16f25423 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:05.992969 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d1475474-20cb-4a1d-a1f8-a18b16f25423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:06.065619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1030/2078] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:06.069229 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f90ca9-0936-4ddf-bb21-f7356a8845d9 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:06.069781 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f90ca9-0936-4ddf-bb21-f7356a8845d9 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:06.072364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f90ca9-0936-4ddf-bb21-f7356a8845d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:06.080128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1475474-20cb-4a1d-a1f8-a18b16f25423 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:06.084846 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364cb6a-6e2b-4245-8685-67c0485c1f1b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a812c3b9b20f44f8814051381a8005b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:06.085764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364cb6a-6e2b-4245-8685-67c0485c1f1b service glance] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:06.086104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364cb6a-6e2b-4245-8685-67c0485c1f1b service glance] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:06.086650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364cb6a-6e2b-4245-8685-67c0485c1f1b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:06.124783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-13f90ca9-0936-4ddf-bb21-f7356a8845d9 None None] MFA Rules not processed for user `3387b03fa72c4e4ab5a2d633ae6a5989`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:06.127807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-13f90ca9-0936-4ddf-bb21-f7356a8845d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:06.173645 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364cb6a-6e2b-4245-8685-67c0485c1f1b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:06.175319 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364cb6a-6e2b-4245-8685-67c0485c1f1b service glance] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:06.187654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1049/2079] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:05 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:32:06.189086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364cb6a-6e2b-4245-8685-67c0485c1f1b service glance] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:06.191355 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6364cb6a-6e2b-4245-8685-67c0485c1f1b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:06.279154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364cb6a-6e2b-4245-8685-67c0485c1f1b service glance] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:06.283902 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c38837-05ff-4aea-a75b-22581bfa320f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:06.285015 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c38837-05ff-4aea-a75b-22581bfa320f service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:06.285777 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c38837-05ff-4aea-a75b-22581bfa320f service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:06.286348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c38837-05ff-4aea-a75b-22581bfa320f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:06.287651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1031/2080] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:06.295498 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f90ca9-0936-4ddf-bb21-f7356a8845d9 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:06.319829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c38837-05ff-4aea-a75b-22581bfa320f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:06.321309 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c38837-05ff-4aea-a75b-22581bfa320f service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:06.339718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1050/2081] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:32:06.342813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f90ca9-0936-4ddf-bb21-f7356a8845d9 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:06.350891 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bae07-0c76-4cf7-abc6-24f45ddecc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:06.351639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bae07-0c76-4cf7-abc6-24f45ddecc5c admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:06.351930 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bae07-0c76-4cf7-abc6-24f45ddecc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:06.352211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bae07-0c76-4cf7-abc6-24f45ddecc5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:06.353417 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47bae07-0c76-4cf7-abc6-24f45ddecc5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:06.356597 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47bae07-0c76-4cf7-abc6-24f45ddecc5c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:06.396091 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1032/2082] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:32:06 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:06.406072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c38837-05ff-4aea-a75b-22581bfa320f service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:06.444626 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c38837-05ff-4aea-a75b-22581bfa320f service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:06.453779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:06.566429 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff8953-ee17-4293-9cfc-a72f3493e628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:06.568546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff8953-ee17-4293-9cfc-a72f3493e628 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:06.569106 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff8953-ee17-4293-9cfc-a72f3493e628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:06.569752 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff8953-ee17-4293-9cfc-a72f3493e628 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:06.571281 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff8953-ee17-4293-9cfc-a72f3493e628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:06.575269 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff8953-ee17-4293-9cfc-a72f3493e628 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:06.631200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1051/2083] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:32:06 2022] POST /identity/v3/users => generated 290 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:06.635205 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bae07-0c76-4cf7-abc6-24f45ddecc5c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:06.684096 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bae07-0c76-4cf7-abc6-24f45ddecc5c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:06.692583 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f05d4-92b7-49d6-a5e4-1dfb52b2935d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:06.694269 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f05d4-92b7-49d6-a5e4-1dfb52b2935d admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:06.695002 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f05d4-92b7-49d6-a5e4-1dfb52b2935d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:06.695492 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f05d4-92b7-49d6-a5e4-1dfb52b2935d admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:06.697330 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f05d4-92b7-49d6-a5e4-1dfb52b2935d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:06.700322 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f05d4-92b7-49d6-a5e4-1dfb52b2935d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:06.738158 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff8953-ee17-4293-9cfc-a72f3493e628 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:06.744565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1033/2084] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:06.760829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff8953-ee17-4293-9cfc-a72f3493e628 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:06.771691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f1bf3-3130-405e-947d-4a9ea73dbbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:06.773027 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f1bf3-3130-405e-947d-4a9ea73dbbae admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:06.773647 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f1bf3-3130-405e-947d-4a9ea73dbbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:06.775802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f1bf3-3130-405e-947d-4a9ea73dbbae admin admin] PATH_INFO: `/v3/projects/010a188b39d04ae59c65c9f3baad59a7/users/c48779a72ec64e2993b71f0671807f6e/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:06.811531 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f1bf3-3130-405e-947d-4a9ea73dbbae admin admin] RBAC: Authorizing `identity:create_grant(project_id=010a188b39d04ae59c65c9f3baad59a7, user_id=c48779a72ec64e2993b71f0671807f6e, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:06.820344 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f1bf3-3130-405e-947d-4a9ea73dbbae admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:06.862640 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1052/2085] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:32:06 2022] PUT /identity/v3/projects/010a188b39d04ae59c65c9f3baad59a7/users/c48779a72ec64e2993b71f0671807f6e/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:32:06.869320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f05d4-92b7-49d6-a5e4-1dfb52b2935d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:06.890125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f1bf3-3130-405e-947d-4a9ea73dbbae admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:06.893895 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-197f1bf3-3130-405e-947d-4a9ea73dbbae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:06.908844 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f05d4-92b7-49d6-a5e4-1dfb52b2935d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:07.075311 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:07.091527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f1bf3-3130-405e-947d-4a9ea73dbbae admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:07.099046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc90f4b-503e-4766-8e81-86e8024e62e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:07.100060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc90f4b-503e-4766-8e81-86e8024e62e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:07.100393 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc90f4b-503e-4766-8e81-86e8024e62e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:07.100854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc90f4b-503e-4766-8e81-86e8024e62e2 admin admin] PATH_INFO: `/v3/projects/63e787de17d9440faf96a0df0649bf2a` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:07.101685 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b46b0-553d-4c16-b906-ab6e756ab0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:07.102329 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc90f4b-503e-4766-8e81-86e8024e62e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63e787de17d9440faf96a0df0649bf2a)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:07.102630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b46b0-553d-4c16-b906-ab6e756ab0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:07.103011 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b46b0-553d-4c16-b906-ab6e756ab0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:07.103432 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b46b0-553d-4c16-b906-ab6e756ab0d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:07.104274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b46b0-553d-4c16-b906-ab6e756ab0d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:07.104461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc90f4b-503e-4766-8e81-86e8024e62e2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:07.106488 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b46b0-553d-4c16-b906-ab6e756ab0d6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:07.119142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1034/2086] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:07.123667 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b46b0-553d-4c16-b906-ab6e756ab0d6 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:07.138670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b46b0-553d-4c16-b906-ab6e756ab0d6 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:07.153312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce32846-363f-4114-bbab-a983e4eaec2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:07.154874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce32846-363f-4114-bbab-a983e4eaec2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:07.157419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce32846-363f-4114-bbab-a983e4eaec2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:07.158087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce32846-363f-4114-bbab-a983e4eaec2b admin admin] PATH_INFO: `/v3/projects/010a188b39d04ae59c65c9f3baad59a7/users/c48779a72ec64e2993b71f0671807f6e/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:07.161530 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce32846-363f-4114-bbab-a983e4eaec2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=010a188b39d04ae59c65c9f3baad59a7, user_id=c48779a72ec64e2993b71f0671807f6e, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:07.166455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce32846-363f-4114-bbab-a983e4eaec2b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:07.183299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1035/2087] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:32:07 2022] PUT /identity/v3/projects/010a188b39d04ae59c65c9f3baad59a7/users/c48779a72ec64e2993b71f0671807f6e/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:07.195453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1bc90f4b-503e-4766-8e81-86e8024e62e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63e787de17d9440faf96a0df0649bf2a', 'request_id': 'req-1bc90f4b-503e-4766-8e81-86e8024e62e2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:07.196566 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e29da-c2ac-4133-931c-4c525b77660b None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:07.196848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e29da-c2ac-4133-931c-4c525b77660b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:07.197401 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e29da-c2ac-4133-931c-4c525b77660b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:07.211169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1bc90f4b-503e-4766-8e81-86e8024e62e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63e787de17d9440faf96a0df0649bf2a', 'request_id': 'req-1bc90f4b-503e-4766-8e81-86e8024e62e2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:07.223473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1bc90f4b-503e-4766-8e81-86e8024e62e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63e787de17d9440faf96a0df0649bf2a', 'request_id': 'req-1bc90f4b-503e-4766-8e81-86e8024e62e2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:07.242298 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1bc90f4b-503e-4766-8e81-86e8024e62e2 admin admin] The token cache is being invalidate because project 63e787de17d9440faf96a0df0649bf2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:07.242701 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1bc90f4b-503e-4766-8e81-86e8024e62e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:07.243394 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1bc90f4b-503e-4766-8e81-86e8024e62e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:07.244311 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d42e29da-c2ac-4133-931c-4c525b77660b None None] MFA Rules not processed for user `c48779a72ec64e2993b71f0671807f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:07.245721 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d42e29da-c2ac-4133-931c-4c525b77660b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:07.249365 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1053/2088] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:06 2022] DELETE /identity/v3/projects/63e787de17d9440faf96a0df0649bf2a => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:32:07.250873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b20aaa-b05f-4009-9b03-d81ee78a86f7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:07.251164 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b20aaa-b05f-4009-9b03-d81ee78a86f7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:07.251438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b20aaa-b05f-4009-9b03-d81ee78a86f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:07.261710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:07.276543 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-75b20aaa-b05f-4009-9b03-d81ee78a86f7 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:07.278264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-75b20aaa-b05f-4009-9b03-d81ee78a86f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:07.339611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1036/2089] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:07 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:32:07.349748 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1054/2090] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:07.358555 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e29da-c2ac-4133-931c-4c525b77660b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:07.359998 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d42e29da-c2ac-4133-931c-4c525b77660b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:07.360195 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92190bd4-9625-4fb2-b46f-03761b948a1d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:07.360703 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92190bd4-9625-4fb2-b46f-03761b948a1d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:07.361136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92190bd4-9625-4fb2-b46f-03761b948a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:07.384862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-92190bd4-9625-4fb2-b46f-03761b948a1d None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:07.386122 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-92190bd4-9625-4fb2-b46f-03761b948a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:07.398300 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1055/2091] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:07.402782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92190bd4-9625-4fb2-b46f-03761b948a1d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:07.413626 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92190bd4-9625-4fb2-b46f-03761b948a1d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:07.418560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697acd3-9cbf-4a1b-9346-67523991785e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:07.419265 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697acd3-9cbf-4a1b-9346-67523991785e admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:07.419572 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697acd3-9cbf-4a1b-9346-67523991785e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:07.419839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697acd3-9cbf-4a1b-9346-67523991785e admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:07.420570 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697acd3-9cbf-4a1b-9346-67523991785e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:32:07.420878 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697acd3-9cbf-4a1b-9346-67523991785e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:07.422541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697acd3-9cbf-4a1b-9346-67523991785e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:07.429451 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e29da-c2ac-4133-931c-4c525b77660b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:07.433943 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf2fa0-1133-4d9f-be44-bfb1a2b095fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:07.434667 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf2fa0-1133-4d9f-be44-bfb1a2b095fd service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:07.434973 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf2fa0-1133-4d9f-be44-bfb1a2b095fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:07.435228 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf2fa0-1133-4d9f-be44-bfb1a2b095fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:07.441314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1056/2092] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:32:07.445490 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697acd3-9cbf-4a1b-9346-67523991785e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:07.446071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf2fa0-1133-4d9f-be44-bfb1a2b095fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:07.447536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf2fa0-1133-4d9f-be44-bfb1a2b095fd service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:07.459838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697acd3-9cbf-4a1b-9346-67523991785e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:07.465973 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdcd366-f62f-419a-9823-def9442acb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:07.467322 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1037/2093] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:07 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:32:07.468239 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcd366-f62f-419a-9823-def9442acb69 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:07.468895 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcd366-f62f-419a-9823-def9442acb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:07.469191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcd366-f62f-419a-9823-def9442acb69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:07.470226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdcd366-f62f-419a-9823-def9442acb69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:07.472719 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdcd366-f62f-419a-9823-def9442acb69 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:07.515627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1057/2094] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:32:07 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:32:07.519704 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf2fa0-1133-4d9f-be44-bfb1a2b095fd service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:07.534382 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf2fa0-1133-4d9f-be44-bfb1a2b095fd service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:07.560658 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdcd366-f62f-419a-9823-def9442acb69 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:07.562741 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3bdcd366-f62f-419a-9823-def9442acb69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:07.605124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:07.626176 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153cf70-d001-45a8-a52b-67d6b1d420af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:07.626846 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153cf70-d001-45a8-a52b-67d6b1d420af admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:07.627189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153cf70-d001-45a8-a52b-67d6b1d420af admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:07.627453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153cf70-d001-45a8-a52b-67d6b1d420af admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:07.629023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153cf70-d001-45a8-a52b-67d6b1d420af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:07.631633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153cf70-d001-45a8-a52b-67d6b1d420af admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:07.662321 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1038/2095] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:32:07 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:07.666403 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153cf70-d001-45a8-a52b-67d6b1d420af admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:07.683890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdcd366-f62f-419a-9823-def9442acb69 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:07.685849 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153cf70-d001-45a8-a52b-67d6b1d420af admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:07.688138 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffc3c9-4200-4842-bbf7-ec3a8b13ac9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:07.688869 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffc3c9-4200-4842-bbf7-ec3a8b13ac9f service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:07.689244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffc3c9-4200-4842-bbf7-ec3a8b13ac9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:07.692839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffc3c9-4200-4842-bbf7-ec3a8b13ac9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:07.693782 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e17f23-a17b-412f-807c-2af57a88c4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:07.694687 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e17f23-a17b-412f-807c-2af57a88c4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:07.695133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e17f23-a17b-412f-807c-2af57a88c4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:07.695774 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e17f23-a17b-412f-807c-2af57a88c4b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:07.696493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e17f23-a17b-412f-807c-2af57a88c4b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:07.697721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9cffc3c9-4200-4842-bbf7-ec3a8b13ac9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:07.698157 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e17f23-a17b-412f-807c-2af57a88c4b9 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:07.713975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1039/2096] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:07.716954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e17f23-a17b-412f-807c-2af57a88c4b9 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:07.740897 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e17f23-a17b-412f-807c-2af57a88c4b9 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:07.748964 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af6535-156a-474d-8d6e-17309ccd941f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:07.750935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af6535-156a-474d-8d6e-17309ccd941f admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:07.751307 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af6535-156a-474d-8d6e-17309ccd941f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:07.751788 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af6535-156a-474d-8d6e-17309ccd941f admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:07.753871 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af6535-156a-474d-8d6e-17309ccd941f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:07.759918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af6535-156a-474d-8d6e-17309ccd941f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:07.785177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1040/2097] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:07.788042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af6535-156a-474d-8d6e-17309ccd941f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:07.803709 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cffc3c9-4200-4842-bbf7-ec3a8b13ac9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:07.805965 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cffc3c9-4200-4842-bbf7-ec3a8b13ac9f service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:07.811755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af6535-156a-474d-8d6e-17309ccd941f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:07.820182 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1058/2098] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:32:07.831371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55a8ef-ecff-4afb-a168-1f7a037e3471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:07.833543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55a8ef-ecff-4afb-a168-1f7a037e3471 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:07.834119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55a8ef-ecff-4afb-a168-1f7a037e3471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:07.834565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55a8ef-ecff-4afb-a168-1f7a037e3471 admin admin] PATH_INFO: `/v3/projects/dcddddc5be864cb0974518261a17700c/users/3b7b77ed3c134eabaa094095be31fd5b/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:07.867642 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55a8ef-ecff-4afb-a168-1f7a037e3471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcddddc5be864cb0974518261a17700c, user_id=3b7b77ed3c134eabaa094095be31fd5b, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:07.872386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55a8ef-ecff-4afb-a168-1f7a037e3471 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:07.892136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1041/2099] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:32:07 2022] PUT /identity/v3/projects/dcddddc5be864cb0974518261a17700c/users/3b7b77ed3c134eabaa094095be31fd5b/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:07.903456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08c0ed-4710-4760-9b36-3c5368365df3 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:07.903626 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08c0ed-4710-4760-9b36-3c5368365df3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:07.903912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08c0ed-4710-4760-9b36-3c5368365df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:07.955759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-7c08c0ed-4710-4760-9b36-3c5368365df3 None None] MFA Rules not processed for user `3b7b77ed3c134eabaa094095be31fd5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:07.960058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7c08c0ed-4710-4760-9b36-3c5368365df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:07.969726 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:08.132343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1059/2100] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:32:08.148129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55a8ef-ecff-4afb-a168-1f7a037e3471 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:08.150228 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4e55a8ef-ecff-4afb-a168-1f7a037e3471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:08.306844 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55a8ef-ecff-4afb-a168-1f7a037e3471 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:08.342647 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c64bc-27a4-4035-94da-3e5cf423a158 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a812c3b9b20f44f8814051381a8005b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:08.346174 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c64bc-27a4-4035-94da-3e5cf423a158 service glance] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:08.346957 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c64bc-27a4-4035-94da-3e5cf423a158 service glance] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:08.347743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c64bc-27a4-4035-94da-3e5cf423a158 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:08.376393 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c64bc-27a4-4035-94da-3e5cf423a158 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:08.379227 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c64bc-27a4-4035-94da-3e5cf423a158 service glance] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:08.403700 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1042/2101] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:32:13.701049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08c0ed-4710-4760-9b36-3c5368365df3 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:13.702648 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7c08c0ed-4710-4760-9b36-3c5368365df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:13.810315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08c0ed-4710-4760-9b36-3c5368365df3 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:13.823631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec3642-d77f-4c70-9da5-5b9d561860c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:13.825106 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec3642-d77f-4c70-9da5-5b9d561860c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:13.825634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec3642-d77f-4c70-9da5-5b9d561860c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:13.826100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec3642-d77f-4c70-9da5-5b9d561860c2 admin admin] PATH_INFO: `/v3/users/3387b03fa72c4e4ab5a2d633ae6a5989` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:13.828241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec3642-d77f-4c70-9da5-5b9d561860c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3387b03fa72c4e4ab5a2d633ae6a5989)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:13.830736 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec3642-d77f-4c70-9da5-5b9d561860c2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:13.902400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f2ec3642-d77f-4c70-9da5-5b9d561860c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3387b03fa72c4e4ab5a2d633ae6a5989'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:13.903225 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f2ec3642-d77f-4c70-9da5-5b9d561860c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3387b03fa72c4e4ab5a2d633ae6a5989'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:13.918824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f2ec3642-d77f-4c70-9da5-5b9d561860c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3387b03fa72c4e4ab5a2d633ae6a5989'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:13.919681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f2ec3642-d77f-4c70-9da5-5b9d561860c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3387b03fa72c4e4ab5a2d633ae6a5989'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:13.930571 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f2ec3642-d77f-4c70-9da5-5b9d561860c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3387b03fa72c4e4ab5a2d633ae6a5989'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:13.952548 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1060/2102] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:32:13 2022] DELETE /identity/v3/users/3387b03fa72c4e4ab5a2d633ae6a5989 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:13.960545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde4ad3-3eaf-4d9e-812c-b5ca19d60016 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:13.961230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde4ad3-3eaf-4d9e-812c-b5ca19d60016 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:13.961767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde4ad3-3eaf-4d9e-812c-b5ca19d60016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:13.985998 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2cde4ad3-3eaf-4d9e-812c-b5ca19d60016 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:13.987385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2cde4ad3-3eaf-4d9e-812c-b5ca19d60016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:14.076443 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1043/2103] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:14.092533 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec3642-d77f-4c70-9da5-5b9d561860c2 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:14.094111 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f2ec3642-d77f-4c70-9da5-5b9d561860c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:14.136376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:14.177767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec3642-d77f-4c70-9da5-5b9d561860c2 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:14.181457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f52ad4-9a4b-4e05-a46a-93e89eae0664 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:14.182136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f52ad4-9a4b-4e05-a46a-93e89eae0664 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:14.182422 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f52ad4-9a4b-4e05-a46a-93e89eae0664 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:14.182705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f52ad4-9a4b-4e05-a46a-93e89eae0664 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:14.193913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f52ad4-9a4b-4e05-a46a-93e89eae0664 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:14.195401 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f52ad4-9a4b-4e05-a46a-93e89eae0664 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:14.206851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1061/2104] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:32:14.752458 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde4ad3-3eaf-4d9e-812c-b5ca19d60016 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:14.754833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2cde4ad3-3eaf-4d9e-812c-b5ca19d60016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:14.779723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde4ad3-3eaf-4d9e-812c-b5ca19d60016 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:14.789442 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f497a41-cd00-47bb-9817-a0f3ffb38f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:14.790786 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f497a41-cd00-47bb-9817-a0f3ffb38f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:14.793976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f497a41-cd00-47bb-9817-a0f3ffb38f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:14.795445 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f497a41-cd00-47bb-9817-a0f3ffb38f86 admin admin] PATH_INFO: `/v3/projects/2ce970851e074039b4c3b010bcaa7b06` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:14.797006 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f497a41-cd00-47bb-9817-a0f3ffb38f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ce970851e074039b4c3b010bcaa7b06)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:14.800414 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f497a41-cd00-47bb-9817-a0f3ffb38f86 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:14.885951 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:14.899987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9f497a41-cd00-47bb-9817-a0f3ffb38f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ce970851e074039b4c3b010bcaa7b06', 'request_id': 'req-9f497a41-cd00-47bb-9817-a0f3ffb38f86'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:14.926861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9f497a41-cd00-47bb-9817-a0f3ffb38f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ce970851e074039b4c3b010bcaa7b06', 'request_id': 'req-9f497a41-cd00-47bb-9817-a0f3ffb38f86'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:14.946410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9f497a41-cd00-47bb-9817-a0f3ffb38f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ce970851e074039b4c3b010bcaa7b06', 'request_id': 'req-9f497a41-cd00-47bb-9817-a0f3ffb38f86'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:14.978132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9f497a41-cd00-47bb-9817-a0f3ffb38f86 admin admin] The token cache is being invalidate because project 2ce970851e074039b4c3b010bcaa7b06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:14.978812 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9f497a41-cd00-47bb-9817-a0f3ffb38f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:14.979234 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9f497a41-cd00-47bb-9817-a0f3ffb38f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:14.980992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1044/2105] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:14 2022] DELETE /identity/v3/projects/2ce970851e074039b4c3b010bcaa7b06 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:14.987646 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f52ad4-9a4b-4e05-a46a-93e89eae0664 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:15.000345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f52ad4-9a4b-4e05-a46a-93e89eae0664 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:15.103818 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033241d-2433-47cc-aa74-7a814b366b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:15.105415 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033241d-2433-47cc-aa74-7a814b366b8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:15.106047 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033241d-2433-47cc-aa74-7a814b366b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:15.106390 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033241d-2433-47cc-aa74-7a814b366b8f admin admin] PATH_INFO: `/v3/users/c48779a72ec64e2993b71f0671807f6e` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:15.109503 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033241d-2433-47cc-aa74-7a814b366b8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c48779a72ec64e2993b71f0671807f6e)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:15.114323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033241d-2433-47cc-aa74-7a814b366b8f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:15.193398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3033241d-2433-47cc-aa74-7a814b366b8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c48779a72ec64e2993b71f0671807f6e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:15.194456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3033241d-2433-47cc-aa74-7a814b366b8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c48779a72ec64e2993b71f0671807f6e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:15.219380 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3033241d-2433-47cc-aa74-7a814b366b8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c48779a72ec64e2993b71f0671807f6e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:15.223911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3033241d-2433-47cc-aa74-7a814b366b8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c48779a72ec64e2993b71f0671807f6e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:15.236354 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-3033241d-2433-47cc-aa74-7a814b366b8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c48779a72ec64e2993b71f0671807f6e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:15.255358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1062/2106] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:32:14 2022] DELETE /identity/v3/users/c48779a72ec64e2993b71f0671807f6e => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:15.685254 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033241d-2433-47cc-aa74-7a814b366b8f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:15.685848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f497a41-cd00-47bb-9817-a0f3ffb38f86 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:15.686875 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3033241d-2433-47cc-aa74-7a814b366b8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:15.687665 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9f497a41-cd00-47bb-9817-a0f3ffb38f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:15.720243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f497a41-cd00-47bb-9817-a0f3ffb38f86 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:15.778417 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:15.828939 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033241d-2433-47cc-aa74-7a814b366b8f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:15.840515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1ce87-6e17-456b-9a26-ded823f78b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:15.842407 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1ce87-6e17-456b-9a26-ded823f78b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:15.842711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9520fd9-f2e2-4086-b5fd-ace58b6ba6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:15.842849 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1ce87-6e17-456b-9a26-ded823f78b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:15.843156 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1ce87-6e17-456b-9a26-ded823f78b93 admin admin] PATH_INFO: `/v3/users/91578702a34b4a1d89127c701cfe14eb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:15.844558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e1ce87-6e17-456b-9a26-ded823f78b93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91578702a34b4a1d89127c701cfe14eb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:15.844686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9520fd9-f2e2-4086-b5fd-ace58b6ba6d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:15.844967 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9520fd9-f2e2-4086-b5fd-ace58b6ba6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:15.845332 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9520fd9-f2e2-4086-b5fd-ace58b6ba6d0 admin admin] PATH_INFO: `/v3/projects/010a188b39d04ae59c65c9f3baad59a7` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:15.846800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e1ce87-6e17-456b-9a26-ded823f78b93 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:15.849313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9520fd9-f2e2-4086-b5fd-ace58b6ba6d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=010a188b39d04ae59c65c9f3baad59a7)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:15.852229 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9520fd9-f2e2-4086-b5fd-ace58b6ba6d0 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:15.930171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d9520fd9-f2e2-4086-b5fd-ace58b6ba6d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '010a188b39d04ae59c65c9f3baad59a7', 'request_id': 'req-d9520fd9-f2e2-4086-b5fd-ace58b6ba6d0'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:15.943580 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d9520fd9-f2e2-4086-b5fd-ace58b6ba6d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '010a188b39d04ae59c65c9f3baad59a7', 'request_id': 'req-d9520fd9-f2e2-4086-b5fd-ace58b6ba6d0'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:15.952812 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d9520fd9-f2e2-4086-b5fd-ace58b6ba6d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '010a188b39d04ae59c65c9f3baad59a7', 'request_id': 'req-d9520fd9-f2e2-4086-b5fd-ace58b6ba6d0'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:15.953038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d1e1ce87-6e17-456b-9a26-ded823f78b93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91578702a34b4a1d89127c701cfe14eb'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:15.954089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d1e1ce87-6e17-456b-9a26-ded823f78b93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91578702a34b4a1d89127c701cfe14eb'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:15.970160 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d9520fd9-f2e2-4086-b5fd-ace58b6ba6d0 admin admin] The token cache is being invalidate because project 010a188b39d04ae59c65c9f3baad59a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:15.970438 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d9520fd9-f2e2-4086-b5fd-ace58b6ba6d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:15.972027 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d9520fd9-f2e2-4086-b5fd-ace58b6ba6d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:15.979026 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1045/2107] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:15 2022] DELETE /identity/v3/projects/010a188b39d04ae59c65c9f3baad59a7 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:15.980905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f74d4-e3e7-4790-8a3c-d582db616ecc None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:15.981120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f74d4-e3e7-4790-8a3c-d582db616ecc None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:15.981419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f74d4-e3e7-4790-8a3c-d582db616ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:15.987458 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d1e1ce87-6e17-456b-9a26-ded823f78b93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91578702a34b4a1d89127c701cfe14eb'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:15.992559 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d1e1ce87-6e17-456b-9a26-ded823f78b93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91578702a34b4a1d89127c701cfe14eb'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:16.005478 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d1e1ce87-6e17-456b-9a26-ded823f78b93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91578702a34b4a1d89127c701cfe14eb'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:16.013582 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-474f74d4-e3e7-4790-8a3c-d582db616ecc None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:16.014992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-474f74d4-e3e7-4790-8a3c-d582db616ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:16.040145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1063/2108] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:32:15 2022] DELETE /identity/v3/users/91578702a34b4a1d89127c701cfe14eb => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:16.046852 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf883d66-b09c-49d2-af62-75010caa893a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:16.047860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf883d66-b09c-49d2-af62-75010caa893a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:16.049159 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf883d66-b09c-49d2-af62-75010caa893a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:16.088415 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-cf883d66-b09c-49d2-af62-75010caa893a None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:16.090900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cf883d66-b09c-49d2-af62-75010caa893a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:16.091860 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:16.119579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1046/2109] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:32:16.132056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2cded-50b0-468f-8934-3f23b9d3e325 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:16.132350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2cded-50b0-468f-8934-3f23b9d3e325 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:16.132639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2cded-50b0-468f-8934-3f23b9d3e325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:16.157918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-09b2cded-50b0-468f-8934-3f23b9d3e325 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:16.159750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-09b2cded-50b0-468f-8934-3f23b9d3e325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:16.208229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1064/2110] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:16.222597 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf883d66-b09c-49d2-af62-75010caa893a None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:16.225630 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cf883d66-b09c-49d2-af62-75010caa893a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:16.269237 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1047/2111] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:16.274480 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2cded-50b0-468f-8934-3f23b9d3e325 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:16.291896 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2cded-50b0-468f-8934-3f23b9d3e325 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:16.306951 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99af1c2-aa82-48fb-8e3b-95d4993942b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:16.308196 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf883d66-b09c-49d2-af62-75010caa893a None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:16.308349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99af1c2-aa82-48fb-8e3b-95d4993942b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:16.308972 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99af1c2-aa82-48fb-8e3b-95d4993942b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:16.309471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99af1c2-aa82-48fb-8e3b-95d4993942b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:16.310634 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99af1c2-aa82-48fb-8e3b-95d4993942b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:32:16.311048 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99af1c2-aa82-48fb-8e3b-95d4993942b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:16.312685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2d126-c9ef-47ed-bc9d-2abc0a7f27d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:16.314068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2d126-c9ef-47ed-bc9d-2abc0a7f27d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:16.314203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99af1c2-aa82-48fb-8e3b-95d4993942b2 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:16.314285 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2d126-c9ef-47ed-bc9d-2abc0a7f27d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:16.314285 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2d126-c9ef-47ed-bc9d-2abc0a7f27d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:16.332823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd2d126-c9ef-47ed-bc9d-2abc0a7f27d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:16.332823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd2d126-c9ef-47ed-bc9d-2abc0a7f27d7 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:16.345019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1065/2112] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:32:16.358013 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1048/2113] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:32:16.358538 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2d126-c9ef-47ed-bc9d-2abc0a7f27d7 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:16.368473 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99af1c2-aa82-48fb-8e3b-95d4993942b2 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:16.370693 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d99af1c2-aa82-48fb-8e3b-95d4993942b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:16.376972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2d126-c9ef-47ed-bc9d-2abc0a7f27d7 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:16.383034 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73649b-d122-42b1-a1a7-7e0503a9db46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:16.384434 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73649b-d122-42b1-a1a7-7e0503a9db46 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:16.384719 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73649b-d122-42b1-a1a7-7e0503a9db46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:16.385003 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73649b-d122-42b1-a1a7-7e0503a9db46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:16.385784 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73649b-d122-42b1-a1a7-7e0503a9db46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:16.387374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99af1c2-aa82-48fb-8e3b-95d4993942b2 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:16.387778 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73649b-d122-42b1-a1a7-7e0503a9db46 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:16.393986 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0f239-6a53-4b23-bb34-674b78c506c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:16.394971 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0f239-6a53-4b23-bb34-674b78c506c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:16.395260 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0f239-6a53-4b23-bb34-674b78c506c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:16.395733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0f239-6a53-4b23-bb34-674b78c506c4 admin admin] PATH_INFO: `/v3/users/4386014ca63b4976ac6356dcf26b2ce4` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:16.397366 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:16.398294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0f239-6a53-4b23-bb34-674b78c506c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4386014ca63b4976ac6356dcf26b2ce4)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:16.401147 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0f239-6a53-4b23-bb34-674b78c506c4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:16.425333 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1066/2114] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:32:16 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:16.431326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73649b-d122-42b1-a1a7-7e0503a9db46 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:16.473768 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d5b0f239-6a53-4b23-bb34-674b78c506c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4386014ca63b4976ac6356dcf26b2ce4'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:16.493942 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d5b0f239-6a53-4b23-bb34-674b78c506c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4386014ca63b4976ac6356dcf26b2ce4'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:16.505367 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d5b0f239-6a53-4b23-bb34-674b78c506c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4386014ca63b4976ac6356dcf26b2ce4'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:16.506689 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d5b0f239-6a53-4b23-bb34-674b78c506c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4386014ca63b4976ac6356dcf26b2ce4'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:16.507628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d5b0f239-6a53-4b23-bb34-674b78c506c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4386014ca63b4976ac6356dcf26b2ce4'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:16.533654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73649b-d122-42b1-a1a7-7e0503a9db46 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:16.543259 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1049/2115] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:32:16 2022] DELETE /identity/v3/users/4386014ca63b4976ac6356dcf26b2ce4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:32:16.549346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53f32e-2441-4543-8a61-e4b088d7724f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:16.551788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53f32e-2441-4543-8a61-e4b088d7724f admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:16.552361 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53f32e-2441-4543-8a61-e4b088d7724f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:16.553493 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53f32e-2441-4543-8a61-e4b088d7724f admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:16.554027 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53f32e-2441-4543-8a61-e4b088d7724f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:16.557314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53f32e-2441-4543-8a61-e4b088d7724f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:16.587966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1067/2116] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:32:16 2022] POST /identity/v3/users => generated 290 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:32:16.600728 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0f239-6a53-4b23-bb34-674b78c506c4 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:16.603965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d5b0f239-6a53-4b23-bb34-674b78c506c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:16.727935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:16.732276 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0f239-6a53-4b23-bb34-674b78c506c4 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:16.741423 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489d9e4-32d1-41ee-aa12-187185751227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:16.742621 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489d9e4-32d1-41ee-aa12-187185751227 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:16.743113 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489d9e4-32d1-41ee-aa12-187185751227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:16.743552 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489d9e4-32d1-41ee-aa12-187185751227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:16.744846 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489d9e4-32d1-41ee-aa12-187185751227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:16.747270 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489d9e4-32d1-41ee-aa12-187185751227 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:16.763152 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1050/2117] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:16.767496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53f32e-2441-4543-8a61-e4b088d7724f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:16.780190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53f32e-2441-4543-8a61-e4b088d7724f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:16.787019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972db69f-e455-4e60-8578-2cccbaf558f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:16.787805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972db69f-e455-4e60-8578-2cccbaf558f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:16.788126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972db69f-e455-4e60-8578-2cccbaf558f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:16.788418 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972db69f-e455-4e60-8578-2cccbaf558f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:16.789258 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972db69f-e455-4e60-8578-2cccbaf558f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:16.791027 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972db69f-e455-4e60-8578-2cccbaf558f7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:16.810502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1068/2118] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:16.814455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489d9e4-32d1-41ee-aa12-187185751227 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:16.826051 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489d9e4-32d1-41ee-aa12-187185751227 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:16.832709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e883228-8ed4-4e9e-8a24-424310d8797d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:16.833728 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e883228-8ed4-4e9e-8a24-424310d8797d admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:16.834084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e883228-8ed4-4e9e-8a24-424310d8797d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:16.834377 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e883228-8ed4-4e9e-8a24-424310d8797d admin admin] PATH_INFO: `/v3/projects/03087d4c237c489092a00139012e197e/users/35f511165dab4bbfb7b872ae22d5f58d/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:16.864933 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e883228-8ed4-4e9e-8a24-424310d8797d admin admin] RBAC: Authorizing `identity:create_grant(project_id=03087d4c237c489092a00139012e197e, user_id=35f511165dab4bbfb7b872ae22d5f58d, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:16.871041 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e883228-8ed4-4e9e-8a24-424310d8797d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:16.878531 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972db69f-e455-4e60-8578-2cccbaf558f7 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:16.881114 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-972db69f-e455-4e60-8578-2cccbaf558f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:16.898038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1051/2119] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:32:16 2022] PUT /identity/v3/projects/03087d4c237c489092a00139012e197e/users/35f511165dab4bbfb7b872ae22d5f58d/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:16.900337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972db69f-e455-4e60-8578-2cccbaf558f7 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:16.903964 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e883228-8ed4-4e9e-8a24-424310d8797d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:16.926568 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6e883228-8ed4-4e9e-8a24-424310d8797d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:17.031974 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da648b2-c0fd-44be-be57-a6b271b1f53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:17.032829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da648b2-c0fd-44be-be57-a6b271b1f53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:17.033196 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da648b2-c0fd-44be-be57-a6b271b1f53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:17.033862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da648b2-c0fd-44be-be57-a6b271b1f53a admin admin] PATH_INFO: `/v3/projects/a6d61985c0514e699483e2c02324cdef` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:17.036381 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da648b2-c0fd-44be-be57-a6b271b1f53a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6d61985c0514e699483e2c02324cdef)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:17.039414 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da648b2-c0fd-44be-be57-a6b271b1f53a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:17.039609 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e883228-8ed4-4e9e-8a24-424310d8797d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:17.047874 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:17.049453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:17.049978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:17.050548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8 admin admin] PATH_INFO: `/v3/projects/b34afc12461b4acc9ea58364103e0710` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:17.053003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b34afc12461b4acc9ea58364103e0710)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:17.057019 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:17.124291 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5da648b2-c0fd-44be-be57-a6b271b1f53a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6d61985c0514e699483e2c02324cdef', 'request_id': 'req-5da648b2-c0fd-44be-be57-a6b271b1f53a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:17.145979 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5da648b2-c0fd-44be-be57-a6b271b1f53a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6d61985c0514e699483e2c02324cdef', 'request_id': 'req-5da648b2-c0fd-44be-be57-a6b271b1f53a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:17.156493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b34afc12461b4acc9ea58364103e0710', 'request_id': 'req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:17.169802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5da648b2-c0fd-44be-be57-a6b271b1f53a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6d61985c0514e699483e2c02324cdef', 'request_id': 'req-5da648b2-c0fd-44be-be57-a6b271b1f53a'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:17.176822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b34afc12461b4acc9ea58364103e0710', 'request_id': 'req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:17.189762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:17.193070 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b34afc12461b4acc9ea58364103e0710', 'request_id': 'req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:17.208373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5da648b2-c0fd-44be-be57-a6b271b1f53a admin admin] The token cache is being invalidate because project a6d61985c0514e699483e2c02324cdef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:17.208825 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5da648b2-c0fd-44be-be57-a6b271b1f53a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:17.209640 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5da648b2-c0fd-44be-be57-a6b271b1f53a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:17.211317 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1069/2120] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:16 2022] DELETE /identity/v3/projects/a6d61985c0514e699483e2c02324cdef => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:32:17.215064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2db5aa-dd6a-443f-bed9-40fa8210a538 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:17.215064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2db5aa-dd6a-443f-bed9-40fa8210a538 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:17.215064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2db5aa-dd6a-443f-bed9-40fa8210a538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:17.216857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8 admin admin] The token cache is being invalidate because project b34afc12461b4acc9ea58364103e0710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:17.217327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:17.220771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:17.224734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1052/2121] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:16 2022] DELETE /identity/v3/projects/b34afc12461b4acc9ea58364103e0710 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:17.226033 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:17.229183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:17.250222 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-cc2db5aa-dd6a-443f-bed9-40fa8210a538 None None] MFA Rules not processed for user `35f511165dab4bbfb7b872ae22d5f58d`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:17.251501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cc2db5aa-dd6a-443f-bed9-40fa8210a538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:17.319006 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f7ad8-aacf-42ed-9a75-ea1c1a64f8f8 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:17.324525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a09133-205e-4b5b-923b-13dc60237a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:17.325521 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a09133-205e-4b5b-923b-13dc60237a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:17.325836 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a09133-205e-4b5b-923b-13dc60237a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:17.326264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a09133-205e-4b5b-923b-13dc60237a3a admin admin] PATH_INFO: `/v3/users/ed16dda203ac489f944507036d97e6f3` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:17.327599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a09133-205e-4b5b-923b-13dc60237a3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed16dda203ac489f944507036d97e6f3)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:17.333426 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a09133-205e-4b5b-923b-13dc60237a3a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:17.339522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:17.367487 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1070/2122] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:17.373019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764036e-440c-4b42-9970-0cbfcb09ae6a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:17.373486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764036e-440c-4b42-9970-0cbfcb09ae6a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:17.374043 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764036e-440c-4b42-9970-0cbfcb09ae6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:17.397071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-0764036e-440c-4b42-9970-0cbfcb09ae6a None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:17.398438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0764036e-440c-4b42-9970-0cbfcb09ae6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:17.404627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-02a09133-205e-4b5b-923b-13dc60237a3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed16dda203ac489f944507036d97e6f3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:17.416523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1071/2123] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:17.417226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764036e-440c-4b42-9970-0cbfcb09ae6a None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:17.419170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0764036e-440c-4b42-9970-0cbfcb09ae6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:17.436878 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-02a09133-205e-4b5b-923b-13dc60237a3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed16dda203ac489f944507036d97e6f3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:17.438547 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764036e-440c-4b42-9970-0cbfcb09ae6a None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:17.445913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7706769-74f1-425c-9df4-0e0866628a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:17.446668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7706769-74f1-425c-9df4-0e0866628a3e admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:17.446959 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7706769-74f1-425c-9df4-0e0866628a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:17.447249 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7706769-74f1-425c-9df4-0e0866628a3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:17.447984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7706769-74f1-425c-9df4-0e0866628a3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:17.448955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-02a09133-205e-4b5b-923b-13dc60237a3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed16dda203ac489f944507036d97e6f3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:17.450001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-02a09133-205e-4b5b-923b-13dc60237a3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed16dda203ac489f944507036d97e6f3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:17.450297 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7706769-74f1-425c-9df4-0e0866628a3e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:17.451116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-02a09133-205e-4b5b-923b-13dc60237a3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed16dda203ac489f944507036d97e6f3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:17.470110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1053/2124] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:32:17 2022] DELETE /identity/v3/users/ed16dda203ac489f944507036d97e6f3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:17.471557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f277bbc-3d94-43fa-a348-100e44934eb8 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:17.471875 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f277bbc-3d94-43fa-a348-100e44934eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:17.472502 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f277bbc-3d94-43fa-a348-100e44934eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:17.500970 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1072/2125] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:32:17 2022] POST /identity/v3/projects => generated 381 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:17.505613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7706769-74f1-425c-9df4-0e0866628a3e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:17.508928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a7706769-74f1-425c-9df4-0e0866628a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:17.510849 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3f277bbc-3d94-43fa-a348-100e44934eb8 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:17.511994 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3f277bbc-3d94-43fa-a348-100e44934eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:17.620658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1054/2126] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:17.625884 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f277bbc-3d94-43fa-a348-100e44934eb8 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:17.638604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f277bbc-3d94-43fa-a348-100e44934eb8 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:17.644336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7706769-74f1-425c-9df4-0e0866628a3e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:17.645751 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde137fe-d0af-4c34-97a0-8213272af118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:17.646720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde137fe-d0af-4c34-97a0-8213272af118 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:17.647069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde137fe-d0af-4c34-97a0-8213272af118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:17.647347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde137fe-d0af-4c34-97a0-8213272af118 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:17.648131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde137fe-d0af-4c34-97a0-8213272af118 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:32:17.648492 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde137fe-d0af-4c34-97a0-8213272af118 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:17.650342 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde137fe-d0af-4c34-97a0-8213272af118 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:17.651348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c197fe7-c202-4015-9555-075320ba3bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:17.652054 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c197fe7-c202-4015-9555-075320ba3bbe admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:17.652382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c197fe7-c202-4015-9555-075320ba3bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:17.652846 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c197fe7-c202-4015-9555-075320ba3bbe admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:17.653374 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c197fe7-c202-4015-9555-075320ba3bbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:17.655844 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c197fe7-c202-4015-9555-075320ba3bbe admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:17.667034 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1055/2127] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:32:17.672864 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde137fe-d0af-4c34-97a0-8213272af118 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:17.754970 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde137fe-d0af-4c34-97a0-8213272af118 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:17.763630 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1073/2128] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:32:17 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:32:17.766850 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20ddb3-5dbb-40f6-9177-5c194eed2c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:17.768822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c197fe7-c202-4015-9555-075320ba3bbe admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:17.770172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20ddb3-5dbb-40f6-9177-5c194eed2c3f admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:17.770983 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20ddb3-5dbb-40f6-9177-5c194eed2c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:17.771281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20ddb3-5dbb-40f6-9177-5c194eed2c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:17.772614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20ddb3-5dbb-40f6-9177-5c194eed2c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:17.775717 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20ddb3-5dbb-40f6-9177-5c194eed2c3f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:17.829614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1056/2129] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:32:17 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:32:17.830898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20ddb3-5dbb-40f6-9177-5c194eed2c3f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:17.837555 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ac20ddb3-5dbb-40f6-9177-5c194eed2c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:17.852512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:17.856419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c197fe7-c202-4015-9555-075320ba3bbe admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:17.873724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:17.951269 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20ddb3-5dbb-40f6-9177-5c194eed2c3f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:17.958526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b6a25-34b3-4c57-801e-3e41aea59cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:17.959675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b6a25-34b3-4c57-801e-3e41aea59cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:17.960071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b6a25-34b3-4c57-801e-3e41aea59cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:17.960442 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b6a25-34b3-4c57-801e-3e41aea59cb1 admin admin] PATH_INFO: `/v3/projects/fb4c3654dc4a43f38a37d75f99d56173` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:17.962323 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b6a25-34b3-4c57-801e-3e41aea59cb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb4c3654dc4a43f38a37d75f99d56173)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:17.965983 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b6a25-34b3-4c57-801e-3e41aea59cb1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:17.970812 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d945b7-2c4b-413e-a25b-77ea055179ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:17.971643 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d945b7-2c4b-413e-a25b-77ea055179ec admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:17.972021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d945b7-2c4b-413e-a25b-77ea055179ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:17.972408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d945b7-2c4b-413e-a25b-77ea055179ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:17.973917 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d945b7-2c4b-413e-a25b-77ea055179ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:17.975869 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d945b7-2c4b-413e-a25b-77ea055179ec admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:17.996323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1074/2130] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:17.998104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d945b7-2c4b-413e-a25b-77ea055179ec admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:18.018245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d945b7-2c4b-413e-a25b-77ea055179ec admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:18.034101 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-833b6a25-34b3-4c57-801e-3e41aea59cb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb4c3654dc4a43f38a37d75f99d56173', 'request_id': 'req-833b6a25-34b3-4c57-801e-3e41aea59cb1'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:18.043715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-833b6a25-34b3-4c57-801e-3e41aea59cb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb4c3654dc4a43f38a37d75f99d56173', 'request_id': 'req-833b6a25-34b3-4c57-801e-3e41aea59cb1'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:18.051584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-833b6a25-34b3-4c57-801e-3e41aea59cb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb4c3654dc4a43f38a37d75f99d56173', 'request_id': 'req-833b6a25-34b3-4c57-801e-3e41aea59cb1'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:18.065156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-833b6a25-34b3-4c57-801e-3e41aea59cb1 admin admin] The token cache is being invalidate because project fb4c3654dc4a43f38a37d75f99d56173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:18.065626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-833b6a25-34b3-4c57-801e-3e41aea59cb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:18.067196 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-833b6a25-34b3-4c57-801e-3e41aea59cb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:18.070206 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1057/2131] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:17 2022] DELETE /identity/v3/projects/fb4c3654dc4a43f38a37d75f99d56173 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:18.071609 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b6a25-34b3-4c57-801e-3e41aea59cb1 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:18.077083 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-833b6a25-34b3-4c57-801e-3e41aea59cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:18.126059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5352e-e80b-477b-9bb8-79dbd9fcf60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:18.126854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5352e-e80b-477b-9bb8-79dbd9fcf60e admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:18.126996 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5352e-e80b-477b-9bb8-79dbd9fcf60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:18.127179 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5352e-e80b-477b-9bb8-79dbd9fcf60e admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:18.127895 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5352e-e80b-477b-9bb8-79dbd9fcf60e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:18.130598 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5352e-e80b-477b-9bb8-79dbd9fcf60e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:18.154705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1075/2132] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:32:17 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:18.156215 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57517502-af3e-4d54-9684-52761e2d3276 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:18.156511 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57517502-af3e-4d54-9684-52761e2d3276 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:18.156794 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57517502-af3e-4d54-9684-52761e2d3276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:18.204112 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-57517502-af3e-4d54-9684-52761e2d3276 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:18.205082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b6a25-34b3-4c57-801e-3e41aea59cb1 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:18.205328 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-57517502-af3e-4d54-9684-52761e2d3276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:18.213508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994be08-d183-4a47-a0e4-6861e586fba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:18.214635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1076/2133] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:18.215248 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994be08-d183-4a47-a0e4-6861e586fba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:18.215478 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57517502-af3e-4d54-9684-52761e2d3276 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:18.215941 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994be08-d183-4a47-a0e4-6861e586fba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:18.216487 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994be08-d183-4a47-a0e4-6861e586fba1 admin admin] PATH_INFO: `/v3/projects/2b90795754364ba8af79e843631784b4/users/e7e2134ead5a4e53876e8516687938b1/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:18.225573 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-57517502-af3e-4d54-9684-52761e2d3276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:18.257626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994be08-d183-4a47-a0e4-6861e586fba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b90795754364ba8af79e843631784b4, user_id=e7e2134ead5a4e53876e8516687938b1, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:18.260729 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994be08-d183-4a47-a0e4-6861e586fba1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:18.260931 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57517502-af3e-4d54-9684-52761e2d3276 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:18.270245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6da6f-ce41-4f5e-8549-bdfdbc9e7c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:18.271354 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6da6f-ce41-4f5e-8549-bdfdbc9e7c91 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:18.272064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6da6f-ce41-4f5e-8549-bdfdbc9e7c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:18.272544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6da6f-ce41-4f5e-8549-bdfdbc9e7c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:18.274115 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6da6f-ce41-4f5e-8549-bdfdbc9e7c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:18.277273 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6da6f-ce41-4f5e-8549-bdfdbc9e7c91 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:18.281042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1058/2134] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:32:18 2022] PUT /identity/v3/projects/2b90795754364ba8af79e843631784b4/users/e7e2134ead5a4e53876e8516687938b1/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:18.282997 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702613cf-0c43-4678-86bc-1719d4760885 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:18.283456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702613cf-0c43-4678-86bc-1719d4760885 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:18.283891 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702613cf-0c43-4678-86bc-1719d4760885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:18.309821 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1077/2135] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:18.310372 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6da6f-ce41-4f5e-8549-bdfdbc9e7c91 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:18.328022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-702613cf-0c43-4678-86bc-1719d4760885 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:18.330142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-702613cf-0c43-4678-86bc-1719d4760885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:18.338584 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6da6f-ce41-4f5e-8549-bdfdbc9e7c91 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:18.384247 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:18.436923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1059/2136] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:18.442396 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702613cf-0c43-4678-86bc-1719d4760885 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:18.455869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702613cf-0c43-4678-86bc-1719d4760885 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:18.466425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613126f-372c-4751-a4e8-0142833050a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:18.468945 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613126f-372c-4751-a4e8-0142833050a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:18.469578 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613126f-372c-4751-a4e8-0142833050a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:18.470150 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613126f-372c-4751-a4e8-0142833050a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:18.471676 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613126f-372c-4751-a4e8-0142833050a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:18.474933 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613126f-372c-4751-a4e8-0142833050a3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:18.485221 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d256b8-cc70-4278-9b56-77d0fae8a066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:18.486474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d256b8-cc70-4278-9b56-77d0fae8a066 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:18.486973 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d256b8-cc70-4278-9b56-77d0fae8a066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:18.487422 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d256b8-cc70-4278-9b56-77d0fae8a066 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:18.488633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d256b8-cc70-4278-9b56-77d0fae8a066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:18.491391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d256b8-cc70-4278-9b56-77d0fae8a066 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:18.495139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:18.497519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1060/2137] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:18.499591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613126f-372c-4751-a4e8-0142833050a3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:18.515983 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1078/2138] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:18.516907 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d256b8-cc70-4278-9b56-77d0fae8a066 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:18.527801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613126f-372c-4751-a4e8-0142833050a3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:18.534220 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2128b-0489-4e00-8a98-b7f01068133b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:18.535818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2128b-0489-4e00-8a98-b7f01068133b admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:18.535818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2128b-0489-4e00-8a98-b7f01068133b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:18.536439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2128b-0489-4e00-8a98-b7f01068133b admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:18.537832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2128b-0489-4e00-8a98-b7f01068133b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:32:18.538493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2128b-0489-4e00-8a98-b7f01068133b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:18.540629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d256b8-cc70-4278-9b56-77d0fae8a066 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:18.541893 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2128b-0489-4e00-8a98-b7f01068133b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:18.551656 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2ec6f-727a-43fa-8491-88d5c6c97d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:18.552829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2ec6f-727a-43fa-8491-88d5c6c97d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:18.553332 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2ec6f-727a-43fa-8491-88d5c6c97d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:18.553741 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2ec6f-727a-43fa-8491-88d5c6c97d02 admin admin] PATH_INFO: `/v3/projects/73352489dd5a48d3b08f22690f8377c6/users/92aa4d29131147c29471bc4cd6d0b9ed/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:18.561804 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1061/2139] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:32:18.563213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2128b-0489-4e00-8a98-b7f01068133b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:18.580176 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2128b-0489-4e00-8a98-b7f01068133b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:18.583645 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2ec6f-727a-43fa-8491-88d5c6c97d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73352489dd5a48d3b08f22690f8377c6, user_id=92aa4d29131147c29471bc4cd6d0b9ed, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:18.588476 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd7634-534f-449f-8d1d-1d35e62e5acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:18.589039 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2ec6f-727a-43fa-8491-88d5c6c97d02 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:18.589987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd7634-534f-449f-8d1d-1d35e62e5acf admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:18.590454 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd7634-534f-449f-8d1d-1d35e62e5acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:18.590923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd7634-534f-449f-8d1d-1d35e62e5acf admin admin] PATH_INFO: `/v3/projects/2b90795754364ba8af79e843631784b4/users/e7e2134ead5a4e53876e8516687938b1/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:18.594478 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd7634-534f-449f-8d1d-1d35e62e5acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b90795754364ba8af79e843631784b4, user_id=e7e2134ead5a4e53876e8516687938b1, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:18.598147 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd7634-534f-449f-8d1d-1d35e62e5acf admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:18.611638 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1079/2140] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:32:18 2022] PUT /identity/v3/projects/73352489dd5a48d3b08f22690f8377c6/users/92aa4d29131147c29471bc4cd6d0b9ed/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:18.612716 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1062/2141] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:32:18 2022] PUT /identity/v3/projects/2b90795754364ba8af79e843631784b4/users/e7e2134ead5a4e53876e8516687938b1/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:18.618265 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2ec6f-727a-43fa-8491-88d5c6c97d02 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:18.626207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3aea8-3d1a-4d41-8254-7bfcf4c62ba8 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:18.626522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3aea8-3d1a-4d41-8254-7bfcf4c62ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:18.626912 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3aea8-3d1a-4d41-8254-7bfcf4c62ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:18.665997 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-67f3aea8-3d1a-4d41-8254-7bfcf4c62ba8 None None] MFA Rules not processed for user `92aa4d29131147c29471bc4cd6d0b9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:18.667203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-67f3aea8-3d1a-4d41-8254-7bfcf4c62ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:18.726213 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2ec6f-727a-43fa-8491-88d5c6c97d02 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:18.763846 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1063/2142] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:18.765966 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb86914-c2fb-40af-ab86-d90464e6362f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:18.766792 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb86914-c2fb-40af-ab86-d90464e6362f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:18.767254 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb86914-c2fb-40af-ab86-d90464e6362f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:18.810900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b0bee-76eb-4cf8-b56b-5474b90d5981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:18.811626 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b0bee-76eb-4cf8-b56b-5474b90d5981 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:18.811908 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b0bee-76eb-4cf8-b56b-5474b90d5981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:18.812186 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b0bee-76eb-4cf8-b56b-5474b90d5981 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:18.812965 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b0bee-76eb-4cf8-b56b-5474b90d5981 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:18.814374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-bfb86914-c2fb-40af-ab86-d90464e6362f None None] MFA Rules not processed for user `e7e2134ead5a4e53876e8516687938b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:18.814984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b0bee-76eb-4cf8-b56b-5474b90d5981 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:18.815623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bfb86914-c2fb-40af-ab86-d90464e6362f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:18.861991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1080/2143] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:32:18 2022] POST /identity/v3/projects => generated 361 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:18.861991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b0bee-76eb-4cf8-b56b-5474b90d5981 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:18.918023 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1064/2144] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:18.918783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb86914-c2fb-40af-ab86-d90464e6362f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:18.936334 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb86914-c2fb-40af-ab86-d90464e6362f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:18.951460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:18.971829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b0bee-76eb-4cf8-b56b-5474b90d5981 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:18.978585 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ffb16-972b-4be6-8062-97f6acfb7d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:18.979521 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ffb16-972b-4be6-8062-97f6acfb7d2a admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:18.979926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ffb16-972b-4be6-8062-97f6acfb7d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:18.980310 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ffb16-972b-4be6-8062-97f6acfb7d2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:18.981225 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ffb16-972b-4be6-8062-97f6acfb7d2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:18.983414 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ffb16-972b-4be6-8062-97f6acfb7d2a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:18.993502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:19.014834 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1081/2145] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:32:18 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:19.015702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ffb16-972b-4be6-8062-97f6acfb7d2a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:19.017617 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-de8ffb16-972b-4be6-8062-97f6acfb7d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:19.032579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e908c55-e45b-43b0-89c1-ec52c53ebdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:19.033494 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e908c55-e45b-43b0-89c1-ec52c53ebdc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:19.033991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e908c55-e45b-43b0-89c1-ec52c53ebdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:19.034367 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e908c55-e45b-43b0-89c1-ec52c53ebdc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:19.035682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e908c55-e45b-43b0-89c1-ec52c53ebdc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:19.038761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e908c55-e45b-43b0-89c1-ec52c53ebdc7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:19.069255 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1065/2146] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:32:18 2022] POST /identity/v3/users => generated 281 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:19.072387 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e908c55-e45b-43b0-89c1-ec52c53ebdc7 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:19.087222 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e908c55-e45b-43b0-89c1-ec52c53ebdc7 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:19.127734 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ffb16-972b-4be6-8062-97f6acfb7d2a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:19.134675 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9ac59-3b4a-4464-9bcb-5e4769b2fc52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:19.135888 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9ac59-3b4a-4464-9bcb-5e4769b2fc52 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:19.136406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9ac59-3b4a-4464-9bcb-5e4769b2fc52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:19.136926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9ac59-3b4a-4464-9bcb-5e4769b2fc52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:19.175058 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15fb55-5c62-4878-bbbc-3f91dfdeb265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:19.176542 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15fb55-5c62-4878-bbbc-3f91dfdeb265 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:19.176937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15fb55-5c62-4878-bbbc-3f91dfdeb265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:19.177430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15fb55-5c62-4878-bbbc-3f91dfdeb265 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:19.178438 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15fb55-5c62-4878-bbbc-3f91dfdeb265 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:19.181243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15fb55-5c62-4878-bbbc-3f91dfdeb265 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:19.206744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1066/2147] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:32:19 2022] POST /identity/v3/users => generated 295 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:19.208250 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15fb55-5c62-4878-bbbc-3f91dfdeb265 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:19.220958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15fb55-5c62-4878-bbbc-3f91dfdeb265 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:19.227299 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc482e5d-ab2e-4549-9eb3-58c0ad5ec600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:19.228079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc482e5d-ab2e-4549-9eb3-58c0ad5ec600 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:19.228369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc482e5d-ab2e-4549-9eb3-58c0ad5ec600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:19.228652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc482e5d-ab2e-4549-9eb3-58c0ad5ec600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:19.229461 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc482e5d-ab2e-4549-9eb3-58c0ad5ec600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:19.231190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc482e5d-ab2e-4549-9eb3-58c0ad5ec600 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:19.233394 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9ac59-3b4a-4464-9bcb-5e4769b2fc52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:19.234900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9ac59-3b4a-4464-9bcb-5e4769b2fc52 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:19.248773 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1067/2148] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:19.248773 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc482e5d-ab2e-4549-9eb3-58c0ad5ec600 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:19.256222 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1082/2149] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:32:19.257459 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9ac59-3b4a-4464-9bcb-5e4769b2fc52 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:19.258451 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc482e5d-ab2e-4549-9eb3-58c0ad5ec600 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:19.263732 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3854203-297c-4dd9-9110-10a8235a0702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:19.264436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3854203-297c-4dd9-9110-10a8235a0702 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:19.264772 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3854203-297c-4dd9-9110-10a8235a0702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:19.265035 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3854203-297c-4dd9-9110-10a8235a0702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:19.266013 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3854203-297c-4dd9-9110-10a8235a0702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:19.268395 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3854203-297c-4dd9-9110-10a8235a0702 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:19.275218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9ac59-3b4a-4464-9bcb-5e4769b2fc52 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:19.283881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfe9d0-a9a8-4632-be9b-cef1bb236105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:19.284771 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfe9d0-a9a8-4632-be9b-cef1bb236105 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:19.285735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfe9d0-a9a8-4632-be9b-cef1bb236105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:19.286361 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1068/2150] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:19.286496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfe9d0-a9a8-4632-be9b-cef1bb236105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:19.287553 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bfe9d0-a9a8-4632-be9b-cef1bb236105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:19.289240 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bfe9d0-a9a8-4632-be9b-cef1bb236105 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:19.290860 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3854203-297c-4dd9-9110-10a8235a0702 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:19.306884 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1083/2151] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:19.308278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3854203-297c-4dd9-9110-10a8235a0702 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:19.311178 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfe9d0-a9a8-4632-be9b-cef1bb236105 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:19.314448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698a0f4-de5b-478a-aea4-8c0cb0940bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:19.315421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698a0f4-de5b-478a-aea4-8c0cb0940bfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:19.315706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698a0f4-de5b-478a-aea4-8c0cb0940bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:19.315983 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698a0f4-de5b-478a-aea4-8c0cb0940bfb admin admin] PATH_INFO: `/v3/projects/b6bfd1b0e9fa485fa9c102664c70d83d/users/2aee91c5155c484f870888d79ba0a57b/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:19.328685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfe9d0-a9a8-4632-be9b-cef1bb236105 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:19.344747 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7ae84-8d70-4806-be95-113aef0dab3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:19.345745 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7ae84-8d70-4806-be95-113aef0dab3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:19.348494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7ae84-8d70-4806-be95-113aef0dab3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:19.348887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7ae84-8d70-4806-be95-113aef0dab3e admin admin] PATH_INFO: `/v3/projects/598e9bd613ef49e398cc7d0698795d1c/users/d8d5f4955a674acab34450b03723ee31/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:19.372473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7ae84-8d70-4806-be95-113aef0dab3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=598e9bd613ef49e398cc7d0698795d1c, user_id=d8d5f4955a674acab34450b03723ee31, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:19.375919 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7ae84-8d70-4806-be95-113aef0dab3e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:19.377163 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698a0f4-de5b-478a-aea4-8c0cb0940bfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6bfd1b0e9fa485fa9c102664c70d83d, user_id=2aee91c5155c484f870888d79ba0a57b, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:19.388401 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698a0f4-de5b-478a-aea4-8c0cb0940bfb admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:19.400222 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1084/2152] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:32:19 2022] PUT /identity/v3/projects/598e9bd613ef49e398cc7d0698795d1c/users/d8d5f4955a674acab34450b03723ee31/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:19.424321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720d4fe-005b-46fe-982f-8c6f13671ce0 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:19.424624 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720d4fe-005b-46fe-982f-8c6f13671ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:19.424994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720d4fe-005b-46fe-982f-8c6f13671ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:19.427033 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1069/2153] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:32:19 2022] PUT /identity/v3/projects/b6bfd1b0e9fa485fa9c102664c70d83d/users/2aee91c5155c484f870888d79ba0a57b/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:32:19.427605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698a0f4-de5b-478a-aea4-8c0cb0940bfb admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:19.462136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f720d4fe-005b-46fe-982f-8c6f13671ce0 None None] MFA Rules not processed for user `d8d5f4955a674acab34450b03723ee31`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:19.463587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f720d4fe-005b-46fe-982f-8c6f13671ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:19.510997 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:19.532740 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698a0f4-de5b-478a-aea4-8c0cb0940bfb admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:19.541097 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:19.541926 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c864e7d-ce02-4aca-b24d-6e34f670b3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:19.542620 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c864e7d-ce02-4aca-b24d-6e34f670b3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:19.542909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c864e7d-ce02-4aca-b24d-6e34f670b3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:19.543189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c864e7d-ce02-4aca-b24d-6e34f670b3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:19.543934 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c864e7d-ce02-4aca-b24d-6e34f670b3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:19.546196 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c864e7d-ce02-4aca-b24d-6e34f670b3b0 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:19.561830 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1070/2154] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:19.569474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c864e7d-ce02-4aca-b24d-6e34f670b3b0 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:19.590420 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c864e7d-ce02-4aca-b24d-6e34f670b3b0 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:19.595662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f8bfb-1ac5-4fe2-8f24-de098b9e6ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:19.597189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f8bfb-1ac5-4fe2-8f24-de098b9e6ca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:19.597710 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f8bfb-1ac5-4fe2-8f24-de098b9e6ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:19.598707 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f8bfb-1ac5-4fe2-8f24-de098b9e6ca5 admin admin] PATH_INFO: `/v3/projects/b6bfd1b0e9fa485fa9c102664c70d83d/users/2aee91c5155c484f870888d79ba0a57b/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:19.601107 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f8bfb-1ac5-4fe2-8f24-de098b9e6ca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6bfd1b0e9fa485fa9c102664c70d83d, user_id=2aee91c5155c484f870888d79ba0a57b, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:19.603875 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f8bfb-1ac5-4fe2-8f24-de098b9e6ca5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:19.612876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1085/2155] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:19 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:32:19.624659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1071/2156] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:32:19 2022] PUT /identity/v3/projects/b6bfd1b0e9fa485fa9c102664c70d83d/users/2aee91c5155c484f870888d79ba0a57b/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:19.639373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb13787-f6dd-4a5b-9a1c-4cf3298c5f34 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:19.639896 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb13787-f6dd-4a5b-9a1c-4cf3298c5f34 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:19.640418 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb13787-f6dd-4a5b-9a1c-4cf3298c5f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:19.663763 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f8bfb-1ac5-4fe2-8f24-de098b9e6ca5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:19.703724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ebb13787-f6dd-4a5b-9a1c-4cf3298c5f34 None None] MFA Rules not processed for user `2aee91c5155c484f870888d79ba0a57b`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:19.706680 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ebb13787-f6dd-4a5b-9a1c-4cf3298c5f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:19.753020 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f8bfb-1ac5-4fe2-8f24-de098b9e6ca5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:19.759238 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b1f55-cb5e-445c-a9a3-08ce4a660e3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:19.765039 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b1f55-cb5e-445c-a9a3-08ce4a660e3a service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:19.765689 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b1f55-cb5e-445c-a9a3-08ce4a660e3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:19.766240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b1f55-cb5e-445c-a9a3-08ce4a660e3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:19.834525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1086/2157] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:19.860404 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b1f55-cb5e-445c-a9a3-08ce4a660e3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:19.862856 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b1f55-cb5e-445c-a9a3-08ce4a660e3a service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:19.879853 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1072/2158] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:19 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:32:19.903142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb13787-f6dd-4a5b-9a1c-4cf3298c5f34 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:19.908090 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b1f55-cb5e-445c-a9a3-08ce4a660e3a service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:19.930856 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b1f55-cb5e-445c-a9a3-08ce4a660e3a service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:19.939275 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba703d2f-12fa-4b82-8152-2fee11973e3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:19.940010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba703d2f-12fa-4b82-8152-2fee11973e3a service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:19.940459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba703d2f-12fa-4b82-8152-2fee11973e3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:19.940752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba703d2f-12fa-4b82-8152-2fee11973e3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:20.022333 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba703d2f-12fa-4b82-8152-2fee11973e3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:20.023817 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba703d2f-12fa-4b82-8152-2fee11973e3a service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:20.036270 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb13787-f6dd-4a5b-9a1c-4cf3298c5f34 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:20.044457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32f38e-4036-4a94-837a-efff01913b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:20.045902 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32f38e-4036-4a94-837a-efff01913b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:20.046362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32f38e-4036-4a94-837a-efff01913b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:20.046617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1073/2159] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:32:20.046861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32f38e-4036-4a94-837a-efff01913b30 admin admin] PATH_INFO: `/v3/users/d8d5f4955a674acab34450b03723ee31` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:20.048377 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba703d2f-12fa-4b82-8152-2fee11973e3a service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:20.049141 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32f38e-4036-4a94-837a-efff01913b30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8d5f4955a674acab34450b03723ee31)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:20.053399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32f38e-4036-4a94-837a-efff01913b30 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:20.074219 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba703d2f-12fa-4b82-8152-2fee11973e3a service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:20.093851 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47657299-3b27-4181-b9d1-7a8d44a205e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:20.096038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47657299-3b27-4181-b9d1-7a8d44a205e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:20.096847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47657299-3b27-4181-b9d1-7a8d44a205e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:20.098826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47657299-3b27-4181-b9d1-7a8d44a205e0 admin admin] PATH_INFO: `/v3/users/35f511165dab4bbfb7b872ae22d5f58d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:20.102860 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47657299-3b27-4181-b9d1-7a8d44a205e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f511165dab4bbfb7b872ae22d5f58d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:20.107157 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47657299-3b27-4181-b9d1-7a8d44a205e0 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:20.134250 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:20.157343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ab32f38e-4036-4a94-837a-efff01913b30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8d5f4955a674acab34450b03723ee31'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:20.158747 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ab32f38e-4036-4a94-837a-efff01913b30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8d5f4955a674acab34450b03723ee31'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:20.194495 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-47657299-3b27-4181-b9d1-7a8d44a205e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f511165dab4bbfb7b872ae22d5f58d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:20.201306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ab32f38e-4036-4a94-837a-efff01913b30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8d5f4955a674acab34450b03723ee31'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:20.202295 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ab32f38e-4036-4a94-837a-efff01913b30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8d5f4955a674acab34450b03723ee31'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:20.204779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:20.217131 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ab32f38e-4036-4a94-837a-efff01913b30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8d5f4955a674acab34450b03723ee31'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:20.222358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-47657299-3b27-4181-b9d1-7a8d44a205e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f511165dab4bbfb7b872ae22d5f58d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:20.241003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-47657299-3b27-4181-b9d1-7a8d44a205e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f511165dab4bbfb7b872ae22d5f58d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:20.242233 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-47657299-3b27-4181-b9d1-7a8d44a205e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f511165dab4bbfb7b872ae22d5f58d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:20.243273 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-47657299-3b27-4181-b9d1-7a8d44a205e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f511165dab4bbfb7b872ae22d5f58d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:20.256312 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1087/2160] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:32:19 2022] DELETE /identity/v3/users/d8d5f4955a674acab34450b03723ee31 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:20.264401 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49657196-930c-483e-b29c-f1bac7f41b74 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:20.264724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49657196-930c-483e-b29c-f1bac7f41b74 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:20.265131 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49657196-930c-483e-b29c-f1bac7f41b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:20.269033 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1074/2161] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:32:20 2022] DELETE /identity/v3/users/35f511165dab4bbfb7b872ae22d5f58d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:20.276736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee1cfd-6ccd-42c7-b3de-3f868942cfb3 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:20.277168 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee1cfd-6ccd-42c7-b3de-3f868942cfb3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:20.277505 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee1cfd-6ccd-42c7-b3de-3f868942cfb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:20.304290 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-93ee1cfd-6ccd-42c7-b3de-3f868942cfb3 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:20.306638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-93ee1cfd-6ccd-42c7-b3de-3f868942cfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:20.310519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-49657196-930c-483e-b29c-f1bac7f41b74 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:20.312216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-49657196-930c-483e-b29c-f1bac7f41b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:20.434489 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1088/2162] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:20.442005 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1075/2163] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:20.446426 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49657196-930c-483e-b29c-f1bac7f41b74 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:20.448059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-49657196-930c-483e-b29c-f1bac7f41b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:20.453578 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee1cfd-6ccd-42c7-b3de-3f868942cfb3 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:20.461179 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-93ee1cfd-6ccd-42c7-b3de-3f868942cfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:20.594060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49657196-930c-483e-b29c-f1bac7f41b74 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:20.601897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f387c3-f6fd-4da1-a9e8-5a1c0ec6ecce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:20.603347 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f387c3-f6fd-4da1-a9e8-5a1c0ec6ecce service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:20.604155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f387c3-f6fd-4da1-a9e8-5a1c0ec6ecce service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:20.604510 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f387c3-f6fd-4da1-a9e8-5a1c0ec6ecce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:20.625585 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f387c3-f6fd-4da1-a9e8-5a1c0ec6ecce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:20.628816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f387c3-f6fd-4da1-a9e8-5a1c0ec6ecce service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:20.637354 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee1cfd-6ccd-42c7-b3de-3f868942cfb3 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:20.646206 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5e611-b3b0-482e-abe9-3b81ef32cf1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:20.647379 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5e611-b3b0-482e-abe9-3b81ef32cf1b service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:20.647688 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1089/2164] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:32:20.648358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5e611-b3b0-482e-abe9-3b81ef32cf1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:20.649059 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5e611-b3b0-482e-abe9-3b81ef32cf1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:20.669696 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd5e611-b3b0-482e-abe9-3b81ef32cf1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:20.672665 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd5e611-b3b0-482e-abe9-3b81ef32cf1b service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:20.694995 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1076/2165] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:32:21.289751 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f387c3-f6fd-4da1-a9e8-5a1c0ec6ecce service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:21.291555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-96f387c3-f6fd-4da1-a9e8-5a1c0ec6ecce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:21.317429 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f387c3-f6fd-4da1-a9e8-5a1c0ec6ecce service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:21.318990 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5e611-b3b0-482e-abe9-3b81ef32cf1b service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:21.321514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-abd5e611-b3b0-482e-abe9-3b81ef32cf1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:21.368191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b8495-83be-401c-807b-ec8670a87169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:21.369503 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b8495-83be-401c-807b-ec8670a87169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:21.369915 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b8495-83be-401c-807b-ec8670a87169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:21.370414 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b8495-83be-401c-807b-ec8670a87169 admin admin] PATH_INFO: `/v3/projects/598e9bd613ef49e398cc7d0698795d1c` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:21.373376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b8495-83be-401c-807b-ec8670a87169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=598e9bd613ef49e398cc7d0698795d1c)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:21.376830 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b8495-83be-401c-807b-ec8670a87169 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:21.381258 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5e611-b3b0-482e-abe9-3b81ef32cf1b service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:21.393859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:21.398043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:21.399028 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:21.399821 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf admin admin] PATH_INFO: `/v3/projects/03087d4c237c489092a00139012e197e` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:21.402325 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf admin admin] RBAC: Authorizing `identity:delete_project(project_id=03087d4c237c489092a00139012e197e)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:21.407014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:21.448789 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:21.471337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-117b8495-83be-401c-807b-ec8670a87169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '598e9bd613ef49e398cc7d0698795d1c', 'request_id': 'req-117b8495-83be-401c-807b-ec8670a87169'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:21.484022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-117b8495-83be-401c-807b-ec8670a87169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '598e9bd613ef49e398cc7d0698795d1c', 'request_id': 'req-117b8495-83be-401c-807b-ec8670a87169'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:21.487946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:INSERT=2,keystone:DELETE=7,keystone:SELECT=91 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:21.498345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-117b8495-83be-401c-807b-ec8670a87169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '598e9bd613ef49e398cc7d0698795d1c', 'request_id': 'req-117b8495-83be-401c-807b-ec8670a87169'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:21.506851 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03087d4c237c489092a00139012e197e', 'request_id': 'req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:21.518070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-117b8495-83be-401c-807b-ec8670a87169 admin admin] The token cache is being invalidate because project 598e9bd613ef49e398cc7d0698795d1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:21.518460 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-117b8495-83be-401c-807b-ec8670a87169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:21.519205 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-117b8495-83be-401c-807b-ec8670a87169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:21.521886 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1090/2166] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:21 2022] DELETE /identity/v3/projects/598e9bd613ef49e398cc7d0698795d1c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:32:21.523457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03087d4c237c489092a00139012e197e', 'request_id': 'req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:21.526886 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1871b4-feda-49a5-ae7a-17dcced6c40f None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:21.527197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1871b4-feda-49a5-ae7a-17dcced6c40f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:21.527465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1871b4-feda-49a5-ae7a-17dcced6c40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:21.537021 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03087d4c237c489092a00139012e197e', 'request_id': 'req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:21.559067 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf admin admin] The token cache is being invalidate because project 03087d4c237c489092a00139012e197e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:21.559381 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:21.561424 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:21.570503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1077/2167] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:21 2022] DELETE /identity/v3/projects/03087d4c237c489092a00139012e197e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:21.570839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-1c1871b4-feda-49a5-ae7a-17dcced6c40f None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:21.572968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1c1871b4-feda-49a5-ae7a-17dcced6c40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:21.575402 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:21.577470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:21.674868 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1091/2168] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:32:21.682954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21c4d3-a33d-4c57-ae96-6eca0743fedf admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:21.687581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42a18b-2a70-4b29-b6c9-a33042238430 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:21.688238 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42a18b-2a70-4b29-b6c9-a33042238430 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:21.688557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42a18b-2a70-4b29-b6c9-a33042238430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:21.688725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9102f-4e41-4aa1-895d-1c1275110018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:21.689658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9102f-4e41-4aa1-895d-1c1275110018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:21.689946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9102f-4e41-4aa1-895d-1c1275110018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:21.690226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9102f-4e41-4aa1-895d-1c1275110018 admin admin] PATH_INFO: `/v3/users/e7e2134ead5a4e53876e8516687938b1` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:21.693888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e9102f-4e41-4aa1-895d-1c1275110018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e2134ead5a4e53876e8516687938b1)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:21.695288 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e9102f-4e41-4aa1-895d-1c1275110018 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:21.729202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ce42a18b-2a70-4b29-b6c9-a33042238430 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:21.730822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ce42a18b-2a70-4b29-b6c9-a33042238430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:21.749341 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1092/2169] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:32:21.756712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42a18b-2a70-4b29-b6c9-a33042238430 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:21.780410 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42a18b-2a70-4b29-b6c9-a33042238430 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:21.789150 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599174cf-a23a-4d40-8c06-9db380210423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:21.790425 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599174cf-a23a-4d40-8c06-9db380210423 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:21.790765 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599174cf-a23a-4d40-8c06-9db380210423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:21.791120 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599174cf-a23a-4d40-8c06-9db380210423 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:21.791958 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599174cf-a23a-4d40-8c06-9db380210423 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:32:21.792645 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599174cf-a23a-4d40-8c06-9db380210423 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:21.795182 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599174cf-a23a-4d40-8c06-9db380210423 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:21.804743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-35e9102f-4e41-4aa1-895d-1c1275110018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e2134ead5a4e53876e8516687938b1'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:21.813720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1093/2170] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:32:21.814892 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599174cf-a23a-4d40-8c06-9db380210423 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:21.825956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599174cf-a23a-4d40-8c06-9db380210423 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:21.830618 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-35e9102f-4e41-4aa1-895d-1c1275110018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e2134ead5a4e53876e8516687938b1'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:21.834912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce2f13-e045-4e57-8549-21c56a5c2c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:21.836348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce2f13-e045-4e57-8549-21c56a5c2c91 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:21.837050 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce2f13-e045-4e57-8549-21c56a5c2c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:21.837842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce2f13-e045-4e57-8549-21c56a5c2c91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:21.838949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce2f13-e045-4e57-8549-21c56a5c2c91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:21.842285 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce2f13-e045-4e57-8549-21c56a5c2c91 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:21.842925 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-35e9102f-4e41-4aa1-895d-1c1275110018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e2134ead5a4e53876e8516687938b1'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:21.843969 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-35e9102f-4e41-4aa1-895d-1c1275110018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e2134ead5a4e53876e8516687938b1'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:21.845263 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-35e9102f-4e41-4aa1-895d-1c1275110018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e2134ead5a4e53876e8516687938b1'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:21.883571 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1078/2171] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:32:21 2022] DELETE /identity/v3/users/e7e2134ead5a4e53876e8516687938b1 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:32:21.907980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9102f-4e41-4aa1-895d-1c1275110018 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:21.912777 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-35e9102f-4e41-4aa1-895d-1c1275110018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:21.915948 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1094/2172] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:32:21 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:21.975540 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:22.056563 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9102f-4e41-4aa1-895d-1c1275110018 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:22.222278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b7809-9feb-4175-a760-68a6fe5658d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:22.223665 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b7809-9feb-4175-a760-68a6fe5658d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:22.223964 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b7809-9feb-4175-a760-68a6fe5658d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:22.224249 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b7809-9feb-4175-a760-68a6fe5658d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:22.224971 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b7809-9feb-4175-a760-68a6fe5658d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:22.227824 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b7809-9feb-4175-a760-68a6fe5658d0 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:22.259653 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1079/2173] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:32:21 2022] POST /identity/v3/users => generated 286 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:22.262869 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce2f13-e045-4e57-8549-21c56a5c2c91 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:22.276365 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce2f13-e045-4e57-8549-21c56a5c2c91 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:22.304951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef583d3-d735-420f-9082-28eb2df45b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:22.305799 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef583d3-d735-420f-9082-28eb2df45b14 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:22.305799 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef583d3-d735-420f-9082-28eb2df45b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:22.306329 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef583d3-d735-420f-9082-28eb2df45b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:22.308092 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef583d3-d735-420f-9082-28eb2df45b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:22.311942 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef583d3-d735-420f-9082-28eb2df45b14 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:22.373837 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1095/2174] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:22.379196 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b7809-9feb-4175-a760-68a6fe5658d0 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:22.405142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b7809-9feb-4175-a760-68a6fe5658d0 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:22.424574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f63994-23f9-48b5-a740-04025c7050c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:22.425424 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f63994-23f9-48b5-a740-04025c7050c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:22.426381 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f63994-23f9-48b5-a740-04025c7050c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:22.426779 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f63994-23f9-48b5-a740-04025c7050c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:22.428233 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f63994-23f9-48b5-a740-04025c7050c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:22.431215 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f63994-23f9-48b5-a740-04025c7050c4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:22.453371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1080/2175] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:22.457093 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef583d3-d735-420f-9082-28eb2df45b14 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:22.480680 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef583d3-d735-420f-9082-28eb2df45b14 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:22.484417 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f63994-23f9-48b5-a740-04025c7050c4 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:22.489853 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-11f63994-23f9-48b5-a740-04025c7050c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:22.495211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6708e-ade4-468d-a659-680603489905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:22.497923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6708e-ade4-468d-a659-680603489905 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:22.498301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6708e-ade4-468d-a659-680603489905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:22.498701 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6708e-ade4-468d-a659-680603489905 admin admin] PATH_INFO: `/v3/projects/af231f2d98544631857757d5b48fd918/users/ce2c7ead49a047c6bc04a783ed0bfc43/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:22.525864 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f63994-23f9-48b5-a740-04025c7050c4 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:22.534740 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26114a3-aa57-46a4-bf48-38e5f7c743d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:22.536873 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26114a3-aa57-46a4-bf48-38e5f7c743d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:22.537195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26114a3-aa57-46a4-bf48-38e5f7c743d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:22.537753 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26114a3-aa57-46a4-bf48-38e5f7c743d7 admin admin] PATH_INFO: `/v3/projects/2b90795754364ba8af79e843631784b4` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:22.538872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26114a3-aa57-46a4-bf48-38e5f7c743d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b90795754364ba8af79e843631784b4)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:22.539817 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6708e-ade4-468d-a659-680603489905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af231f2d98544631857757d5b48fd918, user_id=ce2c7ead49a047c6bc04a783ed0bfc43, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:22.540847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26114a3-aa57-46a4-bf48-38e5f7c743d7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:22.543822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6708e-ade4-468d-a659-680603489905 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:22.559766 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1096/2176] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:32:22 2022] PUT /identity/v3/projects/af231f2d98544631857757d5b48fd918/users/ce2c7ead49a047c6bc04a783ed0bfc43/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:22.565703 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73acae4e-9ce6-4fb0-9c36-535ab1ead2d5 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:22.566002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73acae4e-9ce6-4fb0-9c36-535ab1ead2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:22.566821 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73acae4e-9ce6-4fb0-9c36-535ab1ead2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:22.588737 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-73acae4e-9ce6-4fb0-9c36-535ab1ead2d5 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:22.590007 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-73acae4e-9ce6-4fb0-9c36-535ab1ead2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:22.604754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e26114a3-aa57-46a4-bf48-38e5f7c743d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b90795754364ba8af79e843631784b4', 'request_id': 'req-e26114a3-aa57-46a4-bf48-38e5f7c743d7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:22.617858 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e26114a3-aa57-46a4-bf48-38e5f7c743d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b90795754364ba8af79e843631784b4', 'request_id': 'req-e26114a3-aa57-46a4-bf48-38e5f7c743d7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:22.626810 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e26114a3-aa57-46a4-bf48-38e5f7c743d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b90795754364ba8af79e843631784b4', 'request_id': 'req-e26114a3-aa57-46a4-bf48-38e5f7c743d7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:22.640482 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e26114a3-aa57-46a4-bf48-38e5f7c743d7 admin admin] The token cache is being invalidate because project 2b90795754364ba8af79e843631784b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:22.640774 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e26114a3-aa57-46a4-bf48-38e5f7c743d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:22.641498 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e26114a3-aa57-46a4-bf48-38e5f7c743d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:22.643082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1081/2177] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:22 2022] DELETE /identity/v3/projects/2b90795754364ba8af79e843631784b4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:32:22.649523 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148b581-61c2-45f4-8c8e-630d379d96fc None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:22.649913 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148b581-61c2-45f4-8c8e-630d379d96fc None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:22.650350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148b581-61c2-45f4-8c8e-630d379d96fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:22.673243 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1097/2178] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:32:22.679935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e148b581-61c2-45f4-8c8e-630d379d96fc None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:22.680743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73acae4e-9ce6-4fb0-9c36-535ab1ead2d5 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:22.681719 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e148b581-61c2-45f4-8c8e-630d379d96fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:22.682152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-73acae4e-9ce6-4fb0-9c36-535ab1ead2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:22.690470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1082/2179] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:22.699835 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005ec02-bd7f-46b1-b41c-5ec92922fa80 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:22.699835 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005ec02-bd7f-46b1-b41c-5ec92922fa80 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:22.700488 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005ec02-bd7f-46b1-b41c-5ec92922fa80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:22.712199 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:22.725464 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0005ec02-bd7f-46b1-b41c-5ec92922fa80 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:22.727129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0005ec02-bd7f-46b1-b41c-5ec92922fa80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:22.738544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1083/2180] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:32:22.744200 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005ec02-bd7f-46b1-b41c-5ec92922fa80 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:22.756382 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005ec02-bd7f-46b1-b41c-5ec92922fa80 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:22.761574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251f0b0-cee0-4ebe-969c-98164a2a7272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:22.762278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251f0b0-cee0-4ebe-969c-98164a2a7272 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:22.762564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251f0b0-cee0-4ebe-969c-98164a2a7272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:22.762839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251f0b0-cee0-4ebe-969c-98164a2a7272 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:22.763570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0251f0b0-cee0-4ebe-969c-98164a2a7272 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:32:22.763873 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0251f0b0-cee0-4ebe-969c-98164a2a7272 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:22.765550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0251f0b0-cee0-4ebe-969c-98164a2a7272 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:22.782558 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1084/2181] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:32:22.782765 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73acae4e-9ce6-4fb0-9c36-535ab1ead2d5 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:22.783596 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251f0b0-cee0-4ebe-969c-98164a2a7272 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:22.786468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc696c2e-0b45-4c4d-b37f-3f1f636878e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:22.787177 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc696c2e-0b45-4c4d-b37f-3f1f636878e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:22.787464 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc696c2e-0b45-4c4d-b37f-3f1f636878e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:22.787743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc696c2e-0b45-4c4d-b37f-3f1f636878e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:22.795612 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251f0b0-cee0-4ebe-969c-98164a2a7272 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:22.801249 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc696c2e-0b45-4c4d-b37f-3f1f636878e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:22.801249 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc696c2e-0b45-4c4d-b37f-3f1f636878e5 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:22.803475 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3a0a6-b99e-4baf-bd28-8ae1a62ae67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:22.804194 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3a0a6-b99e-4baf-bd28-8ae1a62ae67e admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:22.804478 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3a0a6-b99e-4baf-bd28-8ae1a62ae67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:22.804755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3a0a6-b99e-4baf-bd28-8ae1a62ae67e admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:22.805492 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3a0a6-b99e-4baf-bd28-8ae1a62ae67e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:22.807493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3a0a6-b99e-4baf-bd28-8ae1a62ae67e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:22.812668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1098/2182] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:32:22.813179 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:22.859313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1085/2183] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:32:22 2022] POST /identity/v3/projects => generated 391 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:22.864787 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc696c2e-0b45-4c4d-b37f-3f1f636878e5 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:22.876703 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc696c2e-0b45-4c4d-b37f-3f1f636878e5 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:22.969588 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f67aff-41a3-42c0-9882-ad15ae1d6a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:22.970764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f67aff-41a3-42c0-9882-ad15ae1d6a55 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:22.971267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f67aff-41a3-42c0-9882-ad15ae1d6a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:22.971770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f67aff-41a3-42c0-9882-ad15ae1d6a55 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:22.975675 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f67aff-41a3-42c0-9882-ad15ae1d6a55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:22.976083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f67aff-41a3-42c0-9882-ad15ae1d6a55 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:23.005892 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1099/2184] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:32:22 2022] POST /identity/v3/users => generated 296 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:23.008364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3a0a6-b99e-4baf-bd28-8ae1a62ae67e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:23.022472 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3a0a6-b99e-4baf-bd28-8ae1a62ae67e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:23.028243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa89188-8cf4-4725-8a9c-ef778baf58c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:23.029053 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa89188-8cf4-4725-8a9c-ef778baf58c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:23.029701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa89188-8cf4-4725-8a9c-ef778baf58c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:23.030188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa89188-8cf4-4725-8a9c-ef778baf58c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:23.031280 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa89188-8cf4-4725-8a9c-ef778baf58c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:23.033944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa89188-8cf4-4725-8a9c-ef778baf58c5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:23.051992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1086/2185] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:23.057583 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f67aff-41a3-42c0-9882-ad15ae1d6a55 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:23.069940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f67aff-41a3-42c0-9882-ad15ae1d6a55 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:23.075975 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18739930-bc5c-4a5a-a708-f809d3bea32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:23.076650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18739930-bc5c-4a5a-a708-f809d3bea32d admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:23.076925 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18739930-bc5c-4a5a-a708-f809d3bea32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:23.077231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18739930-bc5c-4a5a-a708-f809d3bea32d admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:23.078013 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18739930-bc5c-4a5a-a708-f809d3bea32d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:23.079674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18739930-bc5c-4a5a-a708-f809d3bea32d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:23.092669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1100/2186] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:23.098534 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa89188-8cf4-4725-8a9c-ef778baf58c5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:23.118762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa89188-8cf4-4725-8a9c-ef778baf58c5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:23.123774 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d11cdf-432c-42fe-878e-2e8611216fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:23.124677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d11cdf-432c-42fe-878e-2e8611216fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:23.125029 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d11cdf-432c-42fe-878e-2e8611216fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:23.125245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d11cdf-432c-42fe-878e-2e8611216fc3 admin admin] PATH_INFO: `/v3/projects/b148e43339224a52a9fdb3045169af50/users/464cc91d45c94906a78930cec14321e8/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:23.144420 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d11cdf-432c-42fe-878e-2e8611216fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b148e43339224a52a9fdb3045169af50, user_id=464cc91d45c94906a78930cec14321e8, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:23.149452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d11cdf-432c-42fe-878e-2e8611216fc3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:23.167049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1087/2187] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:32:23 2022] PUT /identity/v3/projects/b148e43339224a52a9fdb3045169af50/users/464cc91d45c94906a78930cec14321e8/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:32:23.183283 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c290-3337-4e05-9473-20e3a0ad47d0 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:23.183838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c290-3337-4e05-9473-20e3a0ad47d0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:23.184196 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c290-3337-4e05-9473-20e3a0ad47d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:23.230242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-4b88c290-3337-4e05-9473-20e3a0ad47d0 None None] MFA Rules not processed for user `464cc91d45c94906a78930cec14321e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:23.231705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4b88c290-3337-4e05-9473-20e3a0ad47d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:23.349661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1101/2188] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:23.361048 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d11cdf-432c-42fe-878e-2e8611216fc3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:23.364364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-16d11cdf-432c-42fe-878e-2e8611216fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:23.436634 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d11cdf-432c-42fe-878e-2e8611216fc3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:23.444215 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ab686-fc38-4cb5-a127-ba678c62c9a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:23.445080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ab686-fc38-4cb5-a127-ba678c62c9a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:23.445447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ab686-fc38-4cb5-a127-ba678c62c9a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:23.445832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ab686-fc38-4cb5-a127-ba678c62c9a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:23.458154 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ab686-fc38-4cb5-a127-ba678c62c9a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:23.459794 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ab686-fc38-4cb5-a127-ba678c62c9a2 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:23.472297 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1088/2189] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:32:23.755972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13db84b-c580-460b-9c48-a0cb6d3ff83d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:23.757316 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13db84b-c580-460b-9c48-a0cb6d3ff83d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:23.757844 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13db84b-c580-460b-9c48-a0cb6d3ff83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:23.811064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b13db84b-c580-460b-9c48-a0cb6d3ff83d None None] MFA Rules not processed for user `ce2c7ead49a047c6bc04a783ed0bfc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:23.814692 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b13db84b-c580-460b-9c48-a0cb6d3ff83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:23.832938 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:23.931028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1102/2190] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:23.943277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ab686-fc38-4cb5-a127-ba678c62c9a2 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:23.955633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ab686-fc38-4cb5-a127-ba678c62c9a2 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:23.961190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e167e3c-0807-4dc6-abe0-9c681fdd2041 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:23.963544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e167e3c-0807-4dc6-abe0-9c681fdd2041 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:23.964352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e167e3c-0807-4dc6-abe0-9c681fdd2041 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:23.964483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e167e3c-0807-4dc6-abe0-9c681fdd2041 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:23.975392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e167e3c-0807-4dc6-abe0-9c681fdd2041 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:23.976845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e167e3c-0807-4dc6-abe0-9c681fdd2041 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:23.992274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1089/2191] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:32:26.356773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13db84b-c580-460b-9c48-a0cb6d3ff83d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:26.358482 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b13db84b-c580-460b-9c48-a0cb6d3ff83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:26.370822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13db84b-c580-460b-9c48-a0cb6d3ff83d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:26.374610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7a6ac-8767-4de8-95e4-e7d7ca0603f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:26.375284 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7a6ac-8767-4de8-95e4-e7d7ca0603f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:26.375567 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7a6ac-8767-4de8-95e4-e7d7ca0603f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:26.375844 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7a6ac-8767-4de8-95e4-e7d7ca0603f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:26.377820 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a7a7a6ac-8767-4de8-95e4-e7d7ca0603f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:26.471719 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7a6ac-8767-4de8-95e4-e7d7ca0603f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:26.474326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7a6ac-8767-4de8-95e4-e7d7ca0603f7 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:26.495535 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1103/2192] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:32:26.552976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266671d-0090-49c2-8ecb-9333f7399a43 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:26.553694 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266671d-0090-49c2-8ecb-9333f7399a43 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:26.553694 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266671d-0090-49c2-8ecb-9333f7399a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:26.574564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-9266671d-0090-49c2-8ecb-9333f7399a43 None None] MFA Rules not processed for user `ce2c7ead49a047c6bc04a783ed0bfc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:26.575903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9266671d-0090-49c2-8ecb-9333f7399a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:26.585082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1090/2193] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:36.481011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=51 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:36.567153 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=55,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:55.835684 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7a6ac-8767-4de8-95e4-e7d7ca0603f7 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:55.839059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a7a7a6ac-8767-4de8-95e4-e7d7ca0603f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:56.013956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7a6ac-8767-4de8-95e4-e7d7ca0603f7 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:56.019305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad507f8-09df-44a6-99df-fa956f43837d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:56.020204 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad507f8-09df-44a6-99df-fa956f43837d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:56.020655 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad507f8-09df-44a6-99df-fa956f43837d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:56.020923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad507f8-09df-44a6-99df-fa956f43837d admin admin] PATH_INFO: `/v3/users/3b7b77ed3c134eabaa094095be31fd5b` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:56.022280 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad507f8-09df-44a6-99df-fa956f43837d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b7b77ed3c134eabaa094095be31fd5b)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:56.024352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad507f8-09df-44a6-99df-fa956f43837d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:56.115224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bad507f8-09df-44a6-99df-fa956f43837d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b7b77ed3c134eabaa094095be31fd5b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:56.117469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bad507f8-09df-44a6-99df-fa956f43837d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b7b77ed3c134eabaa094095be31fd5b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:56.138336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bad507f8-09df-44a6-99df-fa956f43837d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b7b77ed3c134eabaa094095be31fd5b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:56.139314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bad507f8-09df-44a6-99df-fa956f43837d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b7b77ed3c134eabaa094095be31fd5b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:56.151451 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bad507f8-09df-44a6-99df-fa956f43837d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b7b77ed3c134eabaa094095be31fd5b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:56.176216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1104/2194] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:32:55 2022] DELETE /identity/v3/users/3b7b77ed3c134eabaa094095be31fd5b => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:56.183113 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d9112-c842-498b-8e98-b970f5e0cf80 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:56.184083 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d9112-c842-498b-8e98-b970f5e0cf80 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:56.186514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d9112-c842-498b-8e98-b970f5e0cf80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:56.216146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2d4d9112-c842-498b-8e98-b970f5e0cf80 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:56.218536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2d4d9112-c842-498b-8e98-b970f5e0cf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:56.353783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1091/2195] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:56.366186 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad507f8-09df-44a6-99df-fa956f43837d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:56.367675 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bad507f8-09df-44a6-99df-fa956f43837d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:56.446561 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad507f8-09df-44a6-99df-fa956f43837d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:56.450226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e543f5-a120-43ff-9ba8-2b3577a59006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:56.450922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e543f5-a120-43ff-9ba8-2b3577a59006 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:56.451206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e543f5-a120-43ff-9ba8-2b3577a59006 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:56.451556 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e543f5-a120-43ff-9ba8-2b3577a59006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:56.463977 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e543f5-a120-43ff-9ba8-2b3577a59006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:56.465713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e543f5-a120-43ff-9ba8-2b3577a59006 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:56.489744 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1105/2196] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:32:56.855709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d9112-c842-498b-8e98-b970f5e0cf80 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:56.857164 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2d4d9112-c842-498b-8e98-b970f5e0cf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:56.870314 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d9112-c842-498b-8e98-b970f5e0cf80 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:56.876579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53eff38-4b62-41dc-adf8-07b00fe2a285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:56.877885 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53eff38-4b62-41dc-adf8-07b00fe2a285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:56.878555 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53eff38-4b62-41dc-adf8-07b00fe2a285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:56.879247 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53eff38-4b62-41dc-adf8-07b00fe2a285 admin admin] PATH_INFO: `/v3/users/464cc91d45c94906a78930cec14321e8` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:56.881230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53eff38-4b62-41dc-adf8-07b00fe2a285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=464cc91d45c94906a78930cec14321e8)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:56.886762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53eff38-4b62-41dc-adf8-07b00fe2a285 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:56.923427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e543f5-a120-43ff-9ba8-2b3577a59006 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:56.926463 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-51e543f5-a120-43ff-9ba8-2b3577a59006 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:56.943421 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:56.950661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e543f5-a120-43ff-9ba8-2b3577a59006 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:56.963646 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2bba7-da43-499e-afa3-dadb693df9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:56.964717 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2bba7-da43-499e-afa3-dadb693df9c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:56.965882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2bba7-da43-499e-afa3-dadb693df9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:56.966669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2bba7-da43-499e-afa3-dadb693df9c9 admin admin] PATH_INFO: `/v3/projects/dcddddc5be864cb0974518261a17700c` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:56.970045 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2bba7-da43-499e-afa3-dadb693df9c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcddddc5be864cb0974518261a17700c)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:56.970243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a53eff38-4b62-41dc-adf8-07b00fe2a285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '464cc91d45c94906a78930cec14321e8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:56.973257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2bba7-da43-499e-afa3-dadb693df9c9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:56.989156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a53eff38-4b62-41dc-adf8-07b00fe2a285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '464cc91d45c94906a78930cec14321e8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:57.005414 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a53eff38-4b62-41dc-adf8-07b00fe2a285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '464cc91d45c94906a78930cec14321e8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:57.006394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a53eff38-4b62-41dc-adf8-07b00fe2a285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '464cc91d45c94906a78930cec14321e8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:57.007254 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a53eff38-4b62-41dc-adf8-07b00fe2a285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '464cc91d45c94906a78930cec14321e8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:57.024643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1092/2197] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:32:56 2022] DELETE /identity/v3/users/464cc91d45c94906a78930cec14321e8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:32:57.030481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff6f50-5831-41cf-9ddc-27714460d2f1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:57.030919 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff6f50-5831-41cf-9ddc-27714460d2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:57.031332 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff6f50-5831-41cf-9ddc-27714460d2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:57.052685 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-84ff6f50-5831-41cf-9ddc-27714460d2f1 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:57.053964 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-84ff6f50-5831-41cf-9ddc-27714460d2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:57.056569 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4fa2bba7-da43-499e-afa3-dadb693df9c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcddddc5be864cb0974518261a17700c', 'request_id': 'req-4fa2bba7-da43-499e-afa3-dadb693df9c9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:57.065215 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4fa2bba7-da43-499e-afa3-dadb693df9c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcddddc5be864cb0974518261a17700c', 'request_id': 'req-4fa2bba7-da43-499e-afa3-dadb693df9c9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:57.087809 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4fa2bba7-da43-499e-afa3-dadb693df9c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcddddc5be864cb0974518261a17700c', 'request_id': 'req-4fa2bba7-da43-499e-afa3-dadb693df9c9'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:57.111219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4fa2bba7-da43-499e-afa3-dadb693df9c9 admin admin] The token cache is being invalidate because project dcddddc5be864cb0974518261a17700c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:57.111749 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4fa2bba7-da43-499e-afa3-dadb693df9c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:57.112867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4fa2bba7-da43-499e-afa3-dadb693df9c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:57.117355 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1106/2198] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:56 2022] DELETE /identity/v3/projects/dcddddc5be864cb0974518261a17700c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:57.122857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe31ea0-f9d1-4418-b3c2-fc699b4375b9 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:57.123490 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe31ea0-f9d1-4418-b3c2-fc699b4375b9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:57.123866 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe31ea0-f9d1-4418-b3c2-fc699b4375b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:57.160932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1093/2199] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:32:57.172692 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff6f50-5831-41cf-9ddc-27714460d2f1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:57.177330 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-84ff6f50-5831-41cf-9ddc-27714460d2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:57.183598 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-abe31ea0-f9d1-4418-b3c2-fc699b4375b9 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:57.185211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-abe31ea0-f9d1-4418-b3c2-fc699b4375b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:57.193329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:57.197016 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1107/2200] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:32:57.209746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1a0c2-6bce-4ed8-b110-a6760667a360 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:57.210353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1a0c2-6bce-4ed8-b110-a6760667a360 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:57.213412 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1a0c2-6bce-4ed8-b110-a6760667a360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:57.239622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-60d1a0c2-6bce-4ed8-b110-a6760667a360 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:57.244534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-60d1a0c2-6bce-4ed8-b110-a6760667a360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:57.251489 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1108/2201] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:32:57.256732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1a0c2-6bce-4ed8-b110-a6760667a360 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:57.272476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1a0c2-6bce-4ed8-b110-a6760667a360 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:57.279897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73327d-1800-4dbe-829d-3dfee3ba39de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:57.281062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73327d-1800-4dbe-829d-3dfee3ba39de admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:57.281835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73327d-1800-4dbe-829d-3dfee3ba39de admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:57.282573 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73327d-1800-4dbe-829d-3dfee3ba39de admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:57.284057 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73327d-1800-4dbe-829d-3dfee3ba39de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:32:57.285529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73327d-1800-4dbe-829d-3dfee3ba39de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:57.285952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff6f50-5831-41cf-9ddc-27714460d2f1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:57.288124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73327d-1800-4dbe-829d-3dfee3ba39de admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:57.299288 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7086e-8dbe-4841-a0bc-bbd0ecba41ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:57.300657 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7086e-8dbe-4841-a0bc-bbd0ecba41ef service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:57.301098 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7086e-8dbe-4841-a0bc-bbd0ecba41ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:57.301098 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7086e-8dbe-4841-a0bc-bbd0ecba41ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:57.307729 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1109/2202] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:32:57.323762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73327d-1800-4dbe-829d-3dfee3ba39de admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:57.349816 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7086e-8dbe-4841-a0bc-bbd0ecba41ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:57.351223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73327d-1800-4dbe-829d-3dfee3ba39de admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:57.351351 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7086e-8dbe-4841-a0bc-bbd0ecba41ef service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:57.356617 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4ff4d-d3e7-473e-b7d5-25094a9a9240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:57.357268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4ff4d-d3e7-473e-b7d5-25094a9a9240 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:57.357622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4ff4d-d3e7-473e-b7d5-25094a9a9240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:57.357897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4ff4d-d3e7-473e-b7d5-25094a9a9240 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:57.358677 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4ff4d-d3e7-473e-b7d5-25094a9a9240 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:57.360803 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4ff4d-d3e7-473e-b7d5-25094a9a9240 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:57.376953 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1094/2203] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:32:57.406012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1110/2204] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:32:57 2022] POST /identity/v3/projects => generated 375 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:57.415081 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7086e-8dbe-4841-a0bc-bbd0ecba41ef service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:57.430038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7086e-8dbe-4841-a0bc-bbd0ecba41ef service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:57.548318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35043175-ad43-439f-a0c0-f9d008821394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:57.550952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35043175-ad43-439f-a0c0-f9d008821394 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:57.550952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35043175-ad43-439f-a0c0-f9d008821394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:57.551746 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35043175-ad43-439f-a0c0-f9d008821394 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:57.553605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35043175-ad43-439f-a0c0-f9d008821394 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:57.559583 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35043175-ad43-439f-a0c0-f9d008821394 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:57.609541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1095/2205] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:32:57 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:32:57.614000 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4ff4d-d3e7-473e-b7d5-25094a9a9240 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:57.638890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4ff4d-d3e7-473e-b7d5-25094a9a9240 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:57.650597 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49738a7e-6acf-4c14-a746-767289768e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:57.651807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49738a7e-6acf-4c14-a746-767289768e23 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:57.652489 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49738a7e-6acf-4c14-a746-767289768e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:57.652987 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49738a7e-6acf-4c14-a746-767289768e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:57.654683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49738a7e-6acf-4c14-a746-767289768e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:57.670051 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49738a7e-6acf-4c14-a746-767289768e23 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:57.687883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1111/2206] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:57.694548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35043175-ad43-439f-a0c0-f9d008821394 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:57.716624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35043175-ad43-439f-a0c0-f9d008821394 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:57.734944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e64be-3715-48a7-8ed0-f09e1dd36f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:57.734944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e64be-3715-48a7-8ed0-f09e1dd36f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:57.734944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e64be-3715-48a7-8ed0-f09e1dd36f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:57.735772 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e64be-3715-48a7-8ed0-f09e1dd36f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:57.737407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e64be-3715-48a7-8ed0-f09e1dd36f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:57.740718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e64be-3715-48a7-8ed0-f09e1dd36f8d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:57.761253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49738a7e-6acf-4c14-a746-767289768e23 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:57.763298 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-49738a7e-6acf-4c14-a746-767289768e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:57.773403 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1096/2207] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:32:57.775259 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e64be-3715-48a7-8ed0-f09e1dd36f8d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:57.788170 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e64be-3715-48a7-8ed0-f09e1dd36f8d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:57.793589 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2abb9-a026-43d9-a687-5b10b70b83e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:57.794537 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2abb9-a026-43d9-a687-5b10b70b83e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:57.794839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2abb9-a026-43d9-a687-5b10b70b83e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:57.795090 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2abb9-a026-43d9-a687-5b10b70b83e5 admin admin] PATH_INFO: `/v3/projects/8beaccf2315245ec8959a58d23ebd293/users/b4d720bb1f5c4336a394c413d9d1c612/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:57.804272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49738a7e-6acf-4c14-a746-767289768e23 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:57.812802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae6444-80c5-4682-9329-5c504d86e7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:57.813738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae6444-80c5-4682-9329-5c504d86e7ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:57.814031 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae6444-80c5-4682-9329-5c504d86e7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:57.814352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae6444-80c5-4682-9329-5c504d86e7ee admin admin] PATH_INFO: `/v3/projects/b148e43339224a52a9fdb3045169af50` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:57.817801 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae6444-80c5-4682-9329-5c504d86e7ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=b148e43339224a52a9fdb3045169af50)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:57.818305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2abb9-a026-43d9-a687-5b10b70b83e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8beaccf2315245ec8959a58d23ebd293, user_id=b4d720bb1f5c4336a394c413d9d1c612, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:57.819780 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae6444-80c5-4682-9329-5c504d86e7ee admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:57.821482 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2abb9-a026-43d9-a687-5b10b70b83e5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:57.846635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1097/2208] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:32:57 2022] PUT /identity/v3/projects/8beaccf2315245ec8959a58d23ebd293/users/b4d720bb1f5c4336a394c413d9d1c612/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:32:57.859512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd824bc9-ab04-41e0-bf32-5a87a65ddcaa None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:57.865665 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd824bc9-ab04-41e0-bf32-5a87a65ddcaa None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:57.866752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd824bc9-ab04-41e0-bf32-5a87a65ddcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:57.907611 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9cae6444-80c5-4682-9329-5c504d86e7ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b148e43339224a52a9fdb3045169af50', 'request_id': 'req-9cae6444-80c5-4682-9329-5c504d86e7ee'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:57.914644 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-fd824bc9-ab04-41e0-bf32-5a87a65ddcaa None None] MFA Rules not processed for user `b4d720bb1f5c4336a394c413d9d1c612`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:32:57.921128 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fd824bc9-ab04-41e0-bf32-5a87a65ddcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:57.921382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9cae6444-80c5-4682-9329-5c504d86e7ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b148e43339224a52a9fdb3045169af50', 'request_id': 'req-9cae6444-80c5-4682-9329-5c504d86e7ee'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:57.939817 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9cae6444-80c5-4682-9329-5c504d86e7ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b148e43339224a52a9fdb3045169af50', 'request_id': 'req-9cae6444-80c5-4682-9329-5c504d86e7ee'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:57.944591 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:57.973716 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9cae6444-80c5-4682-9329-5c504d86e7ee admin admin] The token cache is being invalidate because project b148e43339224a52a9fdb3045169af50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:32:57.974358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9cae6444-80c5-4682-9329-5c504d86e7ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:57.976320 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9cae6444-80c5-4682-9329-5c504d86e7ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:32:57.978965 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1112/2209] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:32:57 2022] DELETE /identity/v3/projects/b148e43339224a52a9fdb3045169af50 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:32:58.032576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:32:58.038660 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1098/2210] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:32:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:32:58.053799 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae6444-80c5-4682-9329-5c504d86e7ee admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:32:58.055842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9cae6444-80c5-4682-9329-5c504d86e7ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:32:58.143134 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae6444-80c5-4682-9329-5c504d86e7ee admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:32:58.149666 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27ead1-ac9d-4450-b120-2bc202284c41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:32:58.150444 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27ead1-ac9d-4450-b120-2bc202284c41 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:32:58.151037 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27ead1-ac9d-4450-b120-2bc202284c41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:32:58.151445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27ead1-ac9d-4450-b120-2bc202284c41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:32:58.163676 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27ead1-ac9d-4450-b120-2bc202284c41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:32:58.165184 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27ead1-ac9d-4450-b120-2bc202284c41 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:32:58.178094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1113/2211] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:32:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:33:03.760094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd824bc9-ab04-41e0-bf32-5a87a65ddcaa None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:03.761861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fd824bc9-ab04-41e0-bf32-5a87a65ddcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:03.875613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd824bc9-ab04-41e0-bf32-5a87a65ddcaa None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:03.886150 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96786a-b8e0-4dc0-b639-9840044b0dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:03.887946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96786a-b8e0-4dc0-b639-9840044b0dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:03.888642 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96786a-b8e0-4dc0-b639-9840044b0dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:03.889346 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96786a-b8e0-4dc0-b639-9840044b0dd1 admin admin] PATH_INFO: `/v3/users/92aa4d29131147c29471bc4cd6d0b9ed` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:03.891765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96786a-b8e0-4dc0-b639-9840044b0dd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92aa4d29131147c29471bc4cd6d0b9ed)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:03.897471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96786a-b8e0-4dc0-b639-9840044b0dd1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:04.004301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5515cd-3b9f-4d44-bda3-9848fb57a477 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:04.005130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5515cd-3b9f-4d44-bda3-9848fb57a477 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:04.005130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5515cd-3b9f-4d44-bda3-9848fb57a477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:04.056893 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-2f5515cd-3b9f-4d44-bda3-9848fb57a477 None None] MFA Rules not processed for user `b4d720bb1f5c4336a394c413d9d1c612`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:04.057999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7a96786a-b8e0-4dc0-b639-9840044b0dd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92aa4d29131147c29471bc4cd6d0b9ed'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:04.059799 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2f5515cd-3b9f-4d44-bda3-9848fb57a477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:04.085142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7a96786a-b8e0-4dc0-b639-9840044b0dd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92aa4d29131147c29471bc4cd6d0b9ed'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:04.093644 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1114/2212] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:33:04.105601 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7a96786a-b8e0-4dc0-b639-9840044b0dd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92aa4d29131147c29471bc4cd6d0b9ed'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:04.106965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7a96786a-b8e0-4dc0-b639-9840044b0dd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92aa4d29131147c29471bc4cd6d0b9ed'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:04.108292 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7a96786a-b8e0-4dc0-b639-9840044b0dd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92aa4d29131147c29471bc4cd6d0b9ed'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:04.143265 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1099/2213] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:33:03 2022] DELETE /identity/v3/users/92aa4d29131147c29471bc4cd6d0b9ed => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:04.150413 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d8011-55c5-4d73-be74-9a65f59bb17a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:04.151421 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d8011-55c5-4d73-be74-9a65f59bb17a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:04.151844 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d8011-55c5-4d73-be74-9a65f59bb17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:04.208391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-560d8011-55c5-4d73-be74-9a65f59bb17a None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:04.210134 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-560d8011-55c5-4d73-be74-9a65f59bb17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:04.332616 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1115/2214] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:33:04.351449 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96786a-b8e0-4dc0-b639-9840044b0dd1 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:04.353045 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7a96786a-b8e0-4dc0-b639-9840044b0dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:04.432198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96786a-b8e0-4dc0-b639-9840044b0dd1 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:04.436812 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0aa6be-b985-40cf-842f-adbf546fec77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:04.437550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0aa6be-b985-40cf-842f-adbf546fec77 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:04.437937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0aa6be-b985-40cf-842f-adbf546fec77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:04.438250 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0aa6be-b985-40cf-842f-adbf546fec77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:04.449774 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0aa6be-b985-40cf-842f-adbf546fec77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:04.451494 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0aa6be-b985-40cf-842f-adbf546fec77 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:04.462977 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1100/2215] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:33:05.219534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d8011-55c5-4d73-be74-9a65f59bb17a None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:05.221166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-560d8011-55c5-4d73-be74-9a65f59bb17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:05.236917 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d8011-55c5-4d73-be74-9a65f59bb17a None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:05.243675 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a10ac-d7c9-4d75-b96c-4a30697e566c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:05.244845 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a10ac-d7c9-4d75-b96c-4a30697e566c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:05.246137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a10ac-d7c9-4d75-b96c-4a30697e566c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:05.246442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a10ac-d7c9-4d75-b96c-4a30697e566c admin admin] PATH_INFO: `/v3/projects/73352489dd5a48d3b08f22690f8377c6` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:05.248042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a10ac-d7c9-4d75-b96c-4a30697e566c admin admin] RBAC: Authorizing `identity:delete_project(project_id=73352489dd5a48d3b08f22690f8377c6)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:05.250095 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a10ac-d7c9-4d75-b96c-4a30697e566c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:05.320111 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-044a10ac-d7c9-4d75-b96c-4a30697e566c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73352489dd5a48d3b08f22690f8377c6', 'request_id': 'req-044a10ac-d7c9-4d75-b96c-4a30697e566c'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:05.327363 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-044a10ac-d7c9-4d75-b96c-4a30697e566c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73352489dd5a48d3b08f22690f8377c6', 'request_id': 'req-044a10ac-d7c9-4d75-b96c-4a30697e566c'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:05.344443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-044a10ac-d7c9-4d75-b96c-4a30697e566c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73352489dd5a48d3b08f22690f8377c6', 'request_id': 'req-044a10ac-d7c9-4d75-b96c-4a30697e566c'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:05.357950 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-044a10ac-d7c9-4d75-b96c-4a30697e566c admin admin] The token cache is being invalidate because project 73352489dd5a48d3b08f22690f8377c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:33:05.358275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-044a10ac-d7c9-4d75-b96c-4a30697e566c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:05.358904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-044a10ac-d7c9-4d75-b96c-4a30697e566c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:05.361488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1116/2216] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:33:05 2022] DELETE /identity/v3/projects/73352489dd5a48d3b08f22690f8377c6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:05.365040 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0aa6be-b985-40cf-842f-adbf546fec77 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:05.376022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0aa6be-b985-40cf-842f-adbf546fec77 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:05.408304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:05.500495 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6274bf1-6638-4c91-a66c-af62cd1dd1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:05.501636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6274bf1-6638-4c91-a66c-af62cd1dd1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:05.502209 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6274bf1-6638-4c91-a66c-af62cd1dd1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:05.503292 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6274bf1-6638-4c91-a66c-af62cd1dd1bf admin admin] PATH_INFO: `/v3/users/2aee91c5155c484f870888d79ba0a57b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:05.505329 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6274bf1-6638-4c91-a66c-af62cd1dd1bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aee91c5155c484f870888d79ba0a57b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:05.507652 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6274bf1-6638-4c91-a66c-af62cd1dd1bf admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:05.579090 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a6274bf1-6638-4c91-a66c-af62cd1dd1bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aee91c5155c484f870888d79ba0a57b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:05.598076 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a6274bf1-6638-4c91-a66c-af62cd1dd1bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aee91c5155c484f870888d79ba0a57b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:05.607547 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a6274bf1-6638-4c91-a66c-af62cd1dd1bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aee91c5155c484f870888d79ba0a57b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:05.613561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a6274bf1-6638-4c91-a66c-af62cd1dd1bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aee91c5155c484f870888d79ba0a57b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:05.613561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a6274bf1-6638-4c91-a66c-af62cd1dd1bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aee91c5155c484f870888d79ba0a57b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:05.630067 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1101/2217] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:33:05 2022] DELETE /identity/v3/users/2aee91c5155c484f870888d79ba0a57b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:05.963737 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a10ac-d7c9-4d75-b96c-4a30697e566c admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:05.965660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-044a10ac-d7c9-4d75-b96c-4a30697e566c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:05.989275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a10ac-d7c9-4d75-b96c-4a30697e566c admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:06.023775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:06.104943 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239078af-c69f-4ee9-9767-fff064393d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:06.106234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239078af-c69f-4ee9-9767-fff064393d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:06.106557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239078af-c69f-4ee9-9767-fff064393d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:06.107052 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239078af-c69f-4ee9-9767-fff064393d62 admin admin] PATH_INFO: `/v3/projects/b6bfd1b0e9fa485fa9c102664c70d83d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:06.108833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239078af-c69f-4ee9-9767-fff064393d62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6bfd1b0e9fa485fa9c102664c70d83d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:06.111179 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239078af-c69f-4ee9-9767-fff064393d62 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:06.191800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-239078af-c69f-4ee9-9767-fff064393d62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6bfd1b0e9fa485fa9c102664c70d83d', 'request_id': 'req-239078af-c69f-4ee9-9767-fff064393d62'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:06.197437 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-239078af-c69f-4ee9-9767-fff064393d62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6bfd1b0e9fa485fa9c102664c70d83d', 'request_id': 'req-239078af-c69f-4ee9-9767-fff064393d62'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:06.204864 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-239078af-c69f-4ee9-9767-fff064393d62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6bfd1b0e9fa485fa9c102664c70d83d', 'request_id': 'req-239078af-c69f-4ee9-9767-fff064393d62'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:06.216321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-239078af-c69f-4ee9-9767-fff064393d62 admin admin] The token cache is being invalidate because project b6bfd1b0e9fa485fa9c102664c70d83d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:33:06.216620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-239078af-c69f-4ee9-9767-fff064393d62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:06.217241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-239078af-c69f-4ee9-9767-fff064393d62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:06.219781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1117/2218] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:33:05 2022] DELETE /identity/v3/projects/b6bfd1b0e9fa485fa9c102664c70d83d => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:06.225355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843df946-c3b5-4d45-b042-8bb6a6b26888 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:06.225654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843df946-c3b5-4d45-b042-8bb6a6b26888 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:06.225934 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843df946-c3b5-4d45-b042-8bb6a6b26888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:06.246193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-843df946-c3b5-4d45-b042-8bb6a6b26888 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:06.247527 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-843df946-c3b5-4d45-b042-8bb6a6b26888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:06.310216 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1102/2219] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:06.318019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71e761-01d3-4a4f-9e17-f6d408e788d7 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:06.318308 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71e761-01d3-4a4f-9e17-f6d408e788d7 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:06.318594 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71e761-01d3-4a4f-9e17-f6d408e788d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:06.338442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-af71e761-01d3-4a4f-9e17-f6d408e788d7 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:06.339685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-af71e761-01d3-4a4f-9e17-f6d408e788d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:06.348344 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1118/2220] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:06.352609 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843df946-c3b5-4d45-b042-8bb6a6b26888 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:06.364323 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843df946-c3b5-4d45-b042-8bb6a6b26888 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:06.372234 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73ec1f-0af4-4949-ad51-f0f711590b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:06.372902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73ec1f-0af4-4949-ad51-f0f711590b35 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:06.373197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73ec1f-0af4-4949-ad51-f0f711590b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:06.373514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73ec1f-0af4-4949-ad51-f0f711590b35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:06.374261 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73ec1f-0af4-4949-ad51-f0f711590b35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:33:06.374693 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73ec1f-0af4-4949-ad51-f0f711590b35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:06.376361 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73ec1f-0af4-4949-ad51-f0f711590b35 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:06.392897 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1103/2221] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:33:06.394210 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71e761-01d3-4a4f-9e17-f6d408e788d7 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:06.405454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71e761-01d3-4a4f-9e17-f6d408e788d7 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:06.411250 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50fdf24-1bb2-4dc6-912d-6f781d915d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:06.411735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fdf24-1bb2-4dc6-912d-6f781d915d3d admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:06.412083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fdf24-1bb2-4dc6-912d-6f781d915d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:06.412393 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fdf24-1bb2-4dc6-912d-6f781d915d3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:06.413227 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50fdf24-1bb2-4dc6-912d-6f781d915d3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:06.415443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50fdf24-1bb2-4dc6-912d-6f781d915d3d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:06.449718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1119/2222] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:33:06 2022] POST /identity/v3/projects => generated 409 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:33:06.460156 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73ec1f-0af4-4949-ad51-f0f711590b35 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:06.484906 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73ec1f-0af4-4949-ad51-f0f711590b35 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:06.529878 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:06.611641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3844e725-ea9c-4c23-a303-8f9710c1070d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:06.613503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844e725-ea9c-4c23-a303-8f9710c1070d admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:06.615532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844e725-ea9c-4c23-a303-8f9710c1070d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:06.615897 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844e725-ea9c-4c23-a303-8f9710c1070d admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:06.617154 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3844e725-ea9c-4c23-a303-8f9710c1070d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:06.621322 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3844e725-ea9c-4c23-a303-8f9710c1070d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:06.655153 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1104/2223] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:33:06 2022] POST /identity/v3/users => generated 305 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:33:06.656059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50fdf24-1bb2-4dc6-912d-6f781d915d3d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:06.672633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50fdf24-1bb2-4dc6-912d-6f781d915d3d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:06.684608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1144989-89ff-4337-ad11-fc529fb2e68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:06.685399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1144989-89ff-4337-ad11-fc529fb2e68e admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:06.685772 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1144989-89ff-4337-ad11-fc529fb2e68e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:06.686203 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1144989-89ff-4337-ad11-fc529fb2e68e admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:06.687023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1144989-89ff-4337-ad11-fc529fb2e68e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:06.689648 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1144989-89ff-4337-ad11-fc529fb2e68e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:06.710381 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1120/2224] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:06.711322 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3844e725-ea9c-4c23-a303-8f9710c1070d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:06.725887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3844e725-ea9c-4c23-a303-8f9710c1070d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:06.753740 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f224b5-6d4d-46e2-92d3-98da4b3017f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:06.755989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f224b5-6d4d-46e2-92d3-98da4b3017f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:06.756290 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f224b5-6d4d-46e2-92d3-98da4b3017f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:06.756572 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f224b5-6d4d-46e2-92d3-98da4b3017f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:06.758332 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f224b5-6d4d-46e2-92d3-98da4b3017f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:06.762598 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f224b5-6d4d-46e2-92d3-98da4b3017f4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:06.795084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1105/2225] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:06.795636 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1144989-89ff-4337-ad11-fc529fb2e68e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:06.810290 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1144989-89ff-4337-ad11-fc529fb2e68e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:06.817696 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6432e8-2b8a-402a-885e-2dc4a10f4ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:06.818844 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6432e8-2b8a-402a-885e-2dc4a10f4ea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:06.819183 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6432e8-2b8a-402a-885e-2dc4a10f4ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:06.819480 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6432e8-2b8a-402a-885e-2dc4a10f4ea8 admin admin] PATH_INFO: `/v3/projects/10e5ac3802134bdc9029e5769579e507/users/32fe3c86586240598357d01c96897265/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:06.848981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6432e8-2b8a-402a-885e-2dc4a10f4ea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e5ac3802134bdc9029e5769579e507, user_id=32fe3c86586240598357d01c96897265, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:06.851773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6432e8-2b8a-402a-885e-2dc4a10f4ea8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:06.867023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1121/2226] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:33:06 2022] PUT /identity/v3/projects/10e5ac3802134bdc9029e5769579e507/users/32fe3c86586240598357d01c96897265/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:06.878526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d4728-e5b1-4a7d-8c9f-a366436e682a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:06.878828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d4728-e5b1-4a7d-8c9f-a366436e682a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:06.879113 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d4728-e5b1-4a7d-8c9f-a366436e682a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:06.912848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-679d4728-e5b1-4a7d-8c9f-a366436e682a None None] MFA Rules not processed for user `32fe3c86586240598357d01c96897265`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:06.913984 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-679d4728-e5b1-4a7d-8c9f-a366436e682a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:07.009901 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1106/2227] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:06 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:07.014710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6432e8-2b8a-402a-885e-2dc4a10f4ea8 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:07.075883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:07.111034 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6432e8-2b8a-402a-885e-2dc4a10f4ea8 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:07.119318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b0264-77b7-4b16-bcad-bf3d0c41bd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:07.120400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b0264-77b7-4b16-bcad-bf3d0c41bd10 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:07.120880 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b0264-77b7-4b16-bcad-bf3d0c41bd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:07.121361 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b0264-77b7-4b16-bcad-bf3d0c41bd10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:07.122487 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b0264-77b7-4b16-bcad-bf3d0c41bd10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:07.125472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b0264-77b7-4b16-bcad-bf3d0c41bd10 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:07.166738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1122/2228] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:33:07 2022] POST /identity/v3/projects => generated 409 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:33:07.167029 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d4728-e5b1-4a7d-8c9f-a366436e682a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:07.183806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d4728-e5b1-4a7d-8c9f-a366436e682a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:07.261954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1045ce6-f7bd-49ca-8986-085be8a6a4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:07.262648 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1045ce6-f7bd-49ca-8986-085be8a6a4ae admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:07.264192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1045ce6-f7bd-49ca-8986-085be8a6a4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:07.264492 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1045ce6-f7bd-49ca-8986-085be8a6a4ae admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:07.265223 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1045ce6-f7bd-49ca-8986-085be8a6a4ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:07.267249 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1045ce6-f7bd-49ca-8986-085be8a6a4ae admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:07.296580 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1107/2229] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:33:07 2022] POST /identity/v3/users => generated 311 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:33:07.301629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b0264-77b7-4b16-bcad-bf3d0c41bd10 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:07.317130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b0264-77b7-4b16-bcad-bf3d0c41bd10 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:07.327131 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91d519-69b9-4b23-a285-a8f08d23895f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:07.328026 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91d519-69b9-4b23-a285-a8f08d23895f admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:07.328362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91d519-69b9-4b23-a285-a8f08d23895f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:07.328713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91d519-69b9-4b23-a285-a8f08d23895f admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:07.329524 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91d519-69b9-4b23-a285-a8f08d23895f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:07.331849 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91d519-69b9-4b23-a285-a8f08d23895f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:07.353334 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1123/2230] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:07.355222 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1045ce6-f7bd-49ca-8986-085be8a6a4ae admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:07.368989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1045ce6-f7bd-49ca-8986-085be8a6a4ae admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:07.375350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a8219-6f31-47e5-b4e9-c07a6e0571b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:07.376330 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a8219-6f31-47e5-b4e9-c07a6e0571b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:07.376625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a8219-6f31-47e5-b4e9-c07a6e0571b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:07.376907 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a8219-6f31-47e5-b4e9-c07a6e0571b6 admin admin] PATH_INFO: `/v3/projects/e8d5838572ec4b17a252b6386ca50dc1/users/41069a3f1aa84109a62f6d6dcd08eab2/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:07.395650 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a8219-6f31-47e5-b4e9-c07a6e0571b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8d5838572ec4b17a252b6386ca50dc1, user_id=41069a3f1aa84109a62f6d6dcd08eab2, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:07.398574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a8219-6f31-47e5-b4e9-c07a6e0571b6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:07.413735 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1108/2231] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:33:07 2022] PUT /identity/v3/projects/e8d5838572ec4b17a252b6386ca50dc1/users/41069a3f1aa84109a62f6d6dcd08eab2/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:07.417840 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91d519-69b9-4b23-a285-a8f08d23895f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:07.429155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91d519-69b9-4b23-a285-a8f08d23895f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:07.512622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c654e-8a4e-4423-af4b-340124f83e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:07.513682 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c654e-8a4e-4423-af4b-340124f83e75 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:07.513682 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c654e-8a4e-4423-af4b-340124f83e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:07.513887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c654e-8a4e-4423-af4b-340124f83e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:07.514634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c654e-8a4e-4423-af4b-340124f83e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:07.516288 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c654e-8a4e-4423-af4b-340124f83e75 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:07.528179 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1124/2232] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:07.533200 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a8219-6f31-47e5-b4e9-c07a6e0571b6 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:07.541126 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:07.547934 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a8219-6f31-47e5-b4e9-c07a6e0571b6 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:07.556791 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9041f9-2872-4c09-b0b6-c6d5bb43a153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:07.557450 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9041f9-2872-4c09-b0b6-c6d5bb43a153 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:07.557739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9041f9-2872-4c09-b0b6-c6d5bb43a153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:07.558025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9041f9-2872-4c09-b0b6-c6d5bb43a153 admin admin] PATH_INFO: `/v3/projects/e8d5838572ec4b17a252b6386ca50dc1/users/41069a3f1aa84109a62f6d6dcd08eab2/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:07.561035 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9041f9-2872-4c09-b0b6-c6d5bb43a153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8d5838572ec4b17a252b6386ca50dc1, user_id=41069a3f1aa84109a62f6d6dcd08eab2, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:07.566428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9041f9-2872-4c09-b0b6-c6d5bb43a153 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:07.582208 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1109/2233] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:33:07 2022] PUT /identity/v3/projects/e8d5838572ec4b17a252b6386ca50dc1/users/41069a3f1aa84109a62f6d6dcd08eab2/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:07.592836 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b1ee5-fed0-4925-b08b-87e2ae48bdfc None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:07.593417 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b1ee5-fed0-4925-b08b-87e2ae48bdfc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:07.593917 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b1ee5-fed0-4925-b08b-87e2ae48bdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:07.631094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e12b1ee5-fed0-4925-b08b-87e2ae48bdfc None None] MFA Rules not processed for user `41069a3f1aa84109a62f6d6dcd08eab2`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:07.632607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e12b1ee5-fed0-4925-b08b-87e2ae48bdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:07.765780 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1125/2234] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:07 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:07.779762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9041f9-2872-4c09-b0b6-c6d5bb43a153 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:07.782118 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5c9041f9-2872-4c09-b0b6-c6d5bb43a153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:07.963425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9041f9-2872-4c09-b0b6-c6d5bb43a153 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:07.980991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f825df-9271-4daf-b52b-c2fc858844b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:07.981778 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f825df-9271-4daf-b52b-c2fc858844b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:07.982102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f825df-9271-4daf-b52b-c2fc858844b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:07.982398 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f825df-9271-4daf-b52b-c2fc858844b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:07.999993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9510a-58e4-40cb-b46e-7a5bdf1a44fd None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:08.000597 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9510a-58e4-40cb-b46e-7a5bdf1a44fd None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:08.001865 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9510a-58e4-40cb-b46e-7a5bdf1a44fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:08.006734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f825df-9271-4daf-b52b-c2fc858844b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:08.008349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f825df-9271-4daf-b52b-c2fc858844b7 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:08.031511 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1110/2235] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:07 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:33:08.058548 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-78d9510a-58e4-40cb-b46e-7a5bdf1a44fd None None] MFA Rules not processed for user `ce2c7ead49a047c6bc04a783ed0bfc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:08.059820 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-78d9510a-58e4-40cb-b46e-7a5bdf1a44fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:08.134956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:08.186154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1126/2236] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:08.298213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f825df-9271-4daf-b52b-c2fc858844b7 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:08.308868 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f825df-9271-4daf-b52b-c2fc858844b7 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:08.312909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d30d2-4603-42ea-b6d6-acedb6dfb56d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:08.313599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d30d2-4603-42ea-b6d6-acedb6dfb56d service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:08.313886 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d30d2-4603-42ea-b6d6-acedb6dfb56d service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:08.314380 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d30d2-4603-42ea-b6d6-acedb6dfb56d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:08.394696 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d30d2-4603-42ea-b6d6-acedb6dfb56d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:08.396314 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d30d2-4603-42ea-b6d6-acedb6dfb56d service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:08.407311 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1111/2237] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:08 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:33:09.139596 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9510a-58e4-40cb-b46e-7a5bdf1a44fd None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:09.144740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-78d9510a-58e4-40cb-b46e-7a5bdf1a44fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:09.242189 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9510a-58e4-40cb-b46e-7a5bdf1a44fd None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:09.251124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97081bc1-1315-40db-aa1c-48b9eb347207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:09.252481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97081bc1-1315-40db-aa1c-48b9eb347207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:09.252481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97081bc1-1315-40db-aa1c-48b9eb347207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:09.253981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97081bc1-1315-40db-aa1c-48b9eb347207 admin admin] PATH_INFO: `/v3/users/d80b4a937e454b98b4a4931d89acac94` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:09.255182 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97081bc1-1315-40db-aa1c-48b9eb347207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d80b4a937e454b98b4a4931d89acac94)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:09.257219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97081bc1-1315-40db-aa1c-48b9eb347207 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:09.329088 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-97081bc1-1315-40db-aa1c-48b9eb347207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd80b4a937e454b98b4a4931d89acac94'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:09.330474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-97081bc1-1315-40db-aa1c-48b9eb347207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd80b4a937e454b98b4a4931d89acac94'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:09.348075 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-97081bc1-1315-40db-aa1c-48b9eb347207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd80b4a937e454b98b4a4931d89acac94'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:09.348814 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-97081bc1-1315-40db-aa1c-48b9eb347207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd80b4a937e454b98b4a4931d89acac94'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:09.356382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-97081bc1-1315-40db-aa1c-48b9eb347207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd80b4a937e454b98b4a4931d89acac94'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:09.370289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1127/2238] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:33:09 2022] DELETE /identity/v3/users/d80b4a937e454b98b4a4931d89acac94 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:09.374766 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba4c38-3ba4-4dde-a5bb-eaea82845cd1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:09.375071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba4c38-3ba4-4dde-a5bb-eaea82845cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:09.375357 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba4c38-3ba4-4dde-a5bb-eaea82845cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:09.395959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-80ba4c38-3ba4-4dde-a5bb-eaea82845cd1 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:09.397206 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-80ba4c38-3ba4-4dde-a5bb-eaea82845cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:09.465838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1112/2239] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:09.476767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97081bc1-1315-40db-aa1c-48b9eb347207 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:09.478251 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-97081bc1-1315-40db-aa1c-48b9eb347207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:09.567781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97081bc1-1315-40db-aa1c-48b9eb347207 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:09.573602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a506139-8645-4997-a9fe-11b2ef7d5a7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:09.574478 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a506139-8645-4997-a9fe-11b2ef7d5a7b service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:09.574873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a506139-8645-4997-a9fe-11b2ef7d5a7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:09.575310 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a506139-8645-4997-a9fe-11b2ef7d5a7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:09.589146 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a506139-8645-4997-a9fe-11b2ef7d5a7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:09.590975 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a506139-8645-4997-a9fe-11b2ef7d5a7b service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:09.606451 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1128/2240] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:33:10.335842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba4c38-3ba4-4dde-a5bb-eaea82845cd1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:10.337915 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-80ba4c38-3ba4-4dde-a5bb-eaea82845cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:10.355100 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba4c38-3ba4-4dde-a5bb-eaea82845cd1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:10.368341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d70309-953b-4bae-899c-df7061d9d58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:10.370364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d70309-953b-4bae-899c-df7061d9d58d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:10.370967 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d70309-953b-4bae-899c-df7061d9d58d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:10.371430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d70309-953b-4bae-899c-df7061d9d58d admin admin] PATH_INFO: `/v3/projects/6e31c7d7676348a3aae0f5060ad557bc` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:10.373419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d70309-953b-4bae-899c-df7061d9d58d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e31c7d7676348a3aae0f5060ad557bc)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:10.376577 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d70309-953b-4bae-899c-df7061d9d58d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:10.387739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:10.462655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e3d70309-953b-4bae-899c-df7061d9d58d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e31c7d7676348a3aae0f5060ad557bc', 'request_id': 'req-e3d70309-953b-4bae-899c-df7061d9d58d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:10.475721 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e3d70309-953b-4bae-899c-df7061d9d58d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e31c7d7676348a3aae0f5060ad557bc', 'request_id': 'req-e3d70309-953b-4bae-899c-df7061d9d58d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:10.487668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e3d70309-953b-4bae-899c-df7061d9d58d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e31c7d7676348a3aae0f5060ad557bc', 'request_id': 'req-e3d70309-953b-4bae-899c-df7061d9d58d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:10.507535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e3d70309-953b-4bae-899c-df7061d9d58d admin admin] The token cache is being invalidate because project 6e31c7d7676348a3aae0f5060ad557bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:33:10.509273 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e3d70309-953b-4bae-899c-df7061d9d58d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:10.510057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-e3d70309-953b-4bae-899c-df7061d9d58d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:10.511858 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1113/2241] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:33:10 2022] DELETE /identity/v3/projects/6e31c7d7676348a3aae0f5060ad557bc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:33:10.519805 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18b587-aea9-47d6-8e3c-620fd3711c34 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:10.520836 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18b587-aea9-47d6-8e3c-620fd3711c34 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:10.521720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18b587-aea9-47d6-8e3c-620fd3711c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:10.531256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:10.549936 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-1d18b587-aea9-47d6-8e3c-620fd3711c34 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:10.551777 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1d18b587-aea9-47d6-8e3c-620fd3711c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:10.627142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1129/2242] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:10.635746 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b32c7-fa93-4a94-b8dd-e65f06075363 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:10.636042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b32c7-fa93-4a94-b8dd-e65f06075363 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:10.636334 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b32c7-fa93-4a94-b8dd-e65f06075363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:10.657936 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-060b32c7-fa93-4a94-b8dd-e65f06075363 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:10.659311 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-060b32c7-fa93-4a94-b8dd-e65f06075363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:10.668009 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1114/2243] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:10.672835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18b587-aea9-47d6-8e3c-620fd3711c34 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:10.692394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54842d7f-447a-4d6f-9d89-7c3051cede65 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:10.692678 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54842d7f-447a-4d6f-9d89-7c3051cede65 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:10.693926 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54842d7f-447a-4d6f-9d89-7c3051cede65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:10.694087 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18b587-aea9-47d6-8e3c-620fd3711c34 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:10.714533 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e959cd0-29c8-4391-9403-ec5a76850394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:10.714533 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e959cd0-29c8-4391-9403-ec5a76850394 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:10.714533 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e959cd0-29c8-4391-9403-ec5a76850394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:10.714533 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e959cd0-29c8-4391-9403-ec5a76850394 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:10.715199 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e959cd0-29c8-4391-9403-ec5a76850394 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:33:10.715199 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e959cd0-29c8-4391-9403-ec5a76850394 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:10.715199 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e959cd0-29c8-4391-9403-ec5a76850394 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:10.715607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-54842d7f-447a-4d6f-9d89-7c3051cede65 None None] MFA Rules not processed for user `cb2bf8531dfe42b29b0b632b39d49625`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:10.715607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-54842d7f-447a-4d6f-9d89-7c3051cede65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:10.730397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1130/2244] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:33:10.741150 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e959cd0-29c8-4391-9403-ec5a76850394 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:10.751924 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e959cd0-29c8-4391-9403-ec5a76850394 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:10.757003 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5ed9a-5889-48cb-83fd-212c7b02ec4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:10.757724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5ed9a-5889-48cb-83fd-212c7b02ec4d admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:10.758044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5ed9a-5889-48cb-83fd-212c7b02ec4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:10.758326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5ed9a-5889-48cb-83fd-212c7b02ec4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:10.759036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5ed9a-5889-48cb-83fd-212c7b02ec4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:10.761020 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5ed9a-5889-48cb-83fd-212c7b02ec4d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:10.807903 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1115/2245] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:33:10.808177 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1131/2246] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:33:10 2022] POST /identity/v3/projects => generated 397 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:33:10.809052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54842d7f-447a-4d6f-9d89-7c3051cede65 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:10.824564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54842d7f-447a-4d6f-9d89-7c3051cede65 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:10.930965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2faf030-01be-4b73-8031-36c958e19b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:10.931991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2faf030-01be-4b73-8031-36c958e19b7b admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:10.932317 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2faf030-01be-4b73-8031-36c958e19b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:10.932694 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2faf030-01be-4b73-8031-36c958e19b7b admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:10.933770 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2faf030-01be-4b73-8031-36c958e19b7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:10.935516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2faf030-01be-4b73-8031-36c958e19b7b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:10.980081 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1116/2247] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:33:10 2022] POST /identity/v3/users => generated 299 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:33:10.982031 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5ed9a-5889-48cb-83fd-212c7b02ec4d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:10.996967 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5ed9a-5889-48cb-83fd-212c7b02ec4d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:11.003041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbbedd-0024-478b-af2e-0fc83ee0d81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:11.003737 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbbedd-0024-478b-af2e-0fc83ee0d81f admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:11.004631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbbedd-0024-478b-af2e-0fc83ee0d81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:11.004631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbbedd-0024-478b-af2e-0fc83ee0d81f admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:11.005055 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbbedd-0024-478b-af2e-0fc83ee0d81f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:11.006800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbbedd-0024-478b-af2e-0fc83ee0d81f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:11.019581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1132/2248] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:33:11.023676 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2faf030-01be-4b73-8031-36c958e19b7b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:11.036655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2faf030-01be-4b73-8031-36c958e19b7b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:11.042920 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170c297-be32-450c-b2f4-f7b7394833e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:11.043611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170c297-be32-450c-b2f4-f7b7394833e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:11.043899 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170c297-be32-450c-b2f4-f7b7394833e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:11.044184 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170c297-be32-450c-b2f4-f7b7394833e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:11.044925 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170c297-be32-450c-b2f4-f7b7394833e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:11.046753 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170c297-be32-450c-b2f4-f7b7394833e5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:11.065508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1117/2249] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:11.068553 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbbedd-0024-478b-af2e-0fc83ee0d81f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:11.085332 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbbedd-0024-478b-af2e-0fc83ee0d81f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:11.087842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178d34b-6332-495d-9e5b-a5f351442d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:11.088759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178d34b-6332-495d-9e5b-a5f351442d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:11.089042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178d34b-6332-495d-9e5b-a5f351442d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:11.089377 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178d34b-6332-495d-9e5b-a5f351442d0a admin admin] PATH_INFO: `/v3/projects/9a72e63c1c1c4b1dbc0bf355c1ff6d63/users/3ec6ca681cf94016a7aeb802469ed8f3/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:11.107992 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178d34b-6332-495d-9e5b-a5f351442d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a72e63c1c1c4b1dbc0bf355c1ff6d63, user_id=3ec6ca681cf94016a7aeb802469ed8f3, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:11.110869 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178d34b-6332-495d-9e5b-a5f351442d0a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:11.139881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1133/2250] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:33:11 2022] PUT /identity/v3/projects/9a72e63c1c1c4b1dbc0bf355c1ff6d63/users/3ec6ca681cf94016a7aeb802469ed8f3/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:11.151594 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d883c67-630d-42fa-89a5-6d14348be767 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:11.151887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d883c67-630d-42fa-89a5-6d14348be767 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:11.152166 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d883c67-630d-42fa-89a5-6d14348be767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:11.174975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:11.185188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1d883c67-630d-42fa-89a5-6d14348be767 None None] MFA Rules not processed for user `3ec6ca681cf94016a7aeb802469ed8f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:11.189088 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1d883c67-630d-42fa-89a5-6d14348be767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:11.288124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1118/2251] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:11.300726 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178d34b-6332-495d-9e5b-a5f351442d0a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:11.307207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6178d34b-6332-495d-9e5b-a5f351442d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:11.397513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178d34b-6332-495d-9e5b-a5f351442d0a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:11.402259 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8984b80e-6642-4829-9f60-7254489c0fc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:11.403112 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984b80e-6642-4829-9f60-7254489c0fc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:11.403239 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984b80e-6642-4829-9f60-7254489c0fc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:11.403492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984b80e-6642-4829-9f60-7254489c0fc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:11.419356 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8984b80e-6642-4829-9f60-7254489c0fc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:11.421652 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8984b80e-6642-4829-9f60-7254489c0fc8 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:11.433599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1134/2252] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:33:18.347247 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d883c67-630d-42fa-89a5-6d14348be767 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:18.349816 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1d883c67-630d-42fa-89a5-6d14348be767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:18.468251 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d883c67-630d-42fa-89a5-6d14348be767 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:18.473872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22299f-ae2b-41e5-b160-81c12792890b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:18.474948 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22299f-ae2b-41e5-b160-81c12792890b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:18.475351 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22299f-ae2b-41e5-b160-81c12792890b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:18.475733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22299f-ae2b-41e5-b160-81c12792890b admin admin] PATH_INFO: `/v3/users/32fe3c86586240598357d01c96897265` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:18.477222 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22299f-ae2b-41e5-b160-81c12792890b admin admin] RBAC: Authorizing `identity:delete_user(user_id=32fe3c86586240598357d01c96897265)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:18.479427 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22299f-ae2b-41e5-b160-81c12792890b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:18.565104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ef22299f-ae2b-41e5-b160-81c12792890b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32fe3c86586240598357d01c96897265'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:18.586824 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ef22299f-ae2b-41e5-b160-81c12792890b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32fe3c86586240598357d01c96897265'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:18.598370 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ef22299f-ae2b-41e5-b160-81c12792890b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32fe3c86586240598357d01c96897265'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:18.599633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ef22299f-ae2b-41e5-b160-81c12792890b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32fe3c86586240598357d01c96897265'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:18.600720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ef22299f-ae2b-41e5-b160-81c12792890b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32fe3c86586240598357d01c96897265'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:18.624170 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1119/2253] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:33:18 2022] DELETE /identity/v3/users/32fe3c86586240598357d01c96897265 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:18.630297 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b26825-e267-4b23-a31b-d40de7672df9 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:18.630849 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b26825-e267-4b23-a31b-d40de7672df9 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:18.631201 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b26825-e267-4b23-a31b-d40de7672df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:18.684463 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-90b26825-e267-4b23-a31b-d40de7672df9 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:18.686331 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-90b26825-e267-4b23-a31b-d40de7672df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:18.738912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:18.779016 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1135/2254] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:18.805561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22299f-ae2b-41e5-b160-81c12792890b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:18.807616 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ef22299f-ae2b-41e5-b160-81c12792890b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:18.872732 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:18.984243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22299f-ae2b-41e5-b160-81c12792890b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:18.997375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285e196-5279-4ab8-b575-9b32c422a5f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:18.998066 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285e196-5279-4ab8-b575-9b32c422a5f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:18.998366 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285e196-5279-4ab8-b575-9b32c422a5f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:18.998661 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285e196-5279-4ab8-b575-9b32c422a5f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:19.029539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a285e196-5279-4ab8-b575-9b32c422a5f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:19.031409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a285e196-5279-4ab8-b575-9b32c422a5f5 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:19.059635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1120/2255] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:33:20.125017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b26825-e267-4b23-a31b-d40de7672df9 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:20.127691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-90b26825-e267-4b23-a31b-d40de7672df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:20.143035 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b26825-e267-4b23-a31b-d40de7672df9 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:20.150659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2010ccd-605d-4815-8c47-fbc495705848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:20.151594 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2010ccd-605d-4815-8c47-fbc495705848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:20.152245 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2010ccd-605d-4815-8c47-fbc495705848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:20.152739 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2010ccd-605d-4815-8c47-fbc495705848 admin admin] PATH_INFO: `/v3/projects/10e5ac3802134bdc9029e5769579e507` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:20.155838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2010ccd-605d-4815-8c47-fbc495705848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10e5ac3802134bdc9029e5769579e507)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:20.158129 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2010ccd-605d-4815-8c47-fbc495705848 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:20.180553 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285e196-5279-4ab8-b575-9b32c422a5f5 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:20.183003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a285e196-5279-4ab8-b575-9b32c422a5f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:20.218762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285e196-5279-4ab8-b575-9b32c422a5f5 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:20.246688 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e2010ccd-605d-4815-8c47-fbc495705848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10e5ac3802134bdc9029e5769579e507', 'request_id': 'req-e2010ccd-605d-4815-8c47-fbc495705848'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:20.258525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e2010ccd-605d-4815-8c47-fbc495705848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10e5ac3802134bdc9029e5769579e507', 'request_id': 'req-e2010ccd-605d-4815-8c47-fbc495705848'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:20.270853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e2010ccd-605d-4815-8c47-fbc495705848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10e5ac3802134bdc9029e5769579e507', 'request_id': 'req-e2010ccd-605d-4815-8c47-fbc495705848'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:20.332605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e2010ccd-605d-4815-8c47-fbc495705848 admin admin] The token cache is being invalidate because project 10e5ac3802134bdc9029e5769579e507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:33:20.332980 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e2010ccd-605d-4815-8c47-fbc495705848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:20.337004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e2010ccd-605d-4815-8c47-fbc495705848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:20.342323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1136/2256] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:33:20 2022] DELETE /identity/v3/projects/10e5ac3802134bdc9029e5769579e507 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:20.344129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84d499-4d2b-4803-a7f2-1fa8c92b80f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:20.345498 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84d499-4d2b-4803-a7f2-1fa8c92b80f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:20.345946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84d499-4d2b-4803-a7f2-1fa8c92b80f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:20.347036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84d499-4d2b-4803-a7f2-1fa8c92b80f6 admin admin] PATH_INFO: `/v3/users/3ec6ca681cf94016a7aeb802469ed8f3` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:20.347156 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2010ccd-605d-4815-8c47-fbc495705848 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:20.349191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84d499-4d2b-4803-a7f2-1fa8c92b80f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ec6ca681cf94016a7aeb802469ed8f3)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:20.352741 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84d499-4d2b-4803-a7f2-1fa8c92b80f6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:20.352941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e2010ccd-605d-4815-8c47-fbc495705848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:20.408202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2010ccd-605d-4815-8c47-fbc495705848 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:20.422287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e1b60-cb09-4ede-9632-50abe2808262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:20.424958 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e1b60-cb09-4ede-9632-50abe2808262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:20.426305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e1b60-cb09-4ede-9632-50abe2808262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:20.427029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e1b60-cb09-4ede-9632-50abe2808262 admin admin] PATH_INFO: `/v3/users/41069a3f1aa84109a62f6d6dcd08eab2` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:20.431277 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e1b60-cb09-4ede-9632-50abe2808262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41069a3f1aa84109a62f6d6dcd08eab2)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:20.436718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e1b60-cb09-4ede-9632-50abe2808262 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:20.453788 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7b84d499-4d2b-4803-a7f2-1fa8c92b80f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ec6ca681cf94016a7aeb802469ed8f3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:20.473359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7b84d499-4d2b-4803-a7f2-1fa8c92b80f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ec6ca681cf94016a7aeb802469ed8f3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:20.485928 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7b84d499-4d2b-4803-a7f2-1fa8c92b80f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ec6ca681cf94016a7aeb802469ed8f3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:20.485928 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7b84d499-4d2b-4803-a7f2-1fa8c92b80f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ec6ca681cf94016a7aeb802469ed8f3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:20.486989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7b84d499-4d2b-4803-a7f2-1fa8c92b80f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ec6ca681cf94016a7aeb802469ed8f3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:20.527589 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1121/2257] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:33:20 2022] DELETE /identity/v3/users/3ec6ca681cf94016a7aeb802469ed8f3 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:33:20.532486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ddb2b-8dcf-42d6-b9cd-9e10015a7f6f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:20.532773 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ddb2b-8dcf-42d6-b9cd-9e10015a7f6f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:20.532985 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ddb2b-8dcf-42d6-b9cd-9e10015a7f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:20.561019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-215e1b60-cb09-4ede-9632-50abe2808262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41069a3f1aa84109a62f6d6dcd08eab2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:20.565330 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-215e1b60-cb09-4ede-9632-50abe2808262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41069a3f1aa84109a62f6d6dcd08eab2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:20.583184 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d40ddb2b-8dcf-42d6-b9cd-9e10015a7f6f None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:20.585783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d40ddb2b-8dcf-42d6-b9cd-9e10015a7f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:20.602867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-215e1b60-cb09-4ede-9632-50abe2808262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41069a3f1aa84109a62f6d6dcd08eab2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:20.604873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-215e1b60-cb09-4ede-9632-50abe2808262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41069a3f1aa84109a62f6d6dcd08eab2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:20.630905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-215e1b60-cb09-4ede-9632-50abe2808262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41069a3f1aa84109a62f6d6dcd08eab2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:20.672774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1137/2258] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:33:20 2022] DELETE /identity/v3/users/41069a3f1aa84109a62f6d6dcd08eab2 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:33:20.717429 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:20.790331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1122/2259] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:20.814906 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e1b60-cb09-4ede-9632-50abe2808262 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:20.818208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-215e1b60-cb09-4ede-9632-50abe2808262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:20.890264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:20.934564 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e1b60-cb09-4ede-9632-50abe2808262 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:20.940859 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d158c2c-a99b-49b3-bb0f-a179b2588a12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:20.943047 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d158c2c-a99b-49b3-bb0f-a179b2588a12 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:20.943660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d158c2c-a99b-49b3-bb0f-a179b2588a12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:20.944327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d158c2c-a99b-49b3-bb0f-a179b2588a12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:21.129307 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d158c2c-a99b-49b3-bb0f-a179b2588a12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:21.130902 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d158c2c-a99b-49b3-bb0f-a179b2588a12 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:21.147977 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1138/2260] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:33:21.574181 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ddb2b-8dcf-42d6-b9cd-9e10015a7f6f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:21.579222 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d40ddb2b-8dcf-42d6-b9cd-9e10015a7f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:21.644094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ddb2b-8dcf-42d6-b9cd-9e10015a7f6f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:21.652467 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce786f-b2ca-4fd8-98bb-13a6c043c02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:21.654094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce786f-b2ca-4fd8-98bb-13a6c043c02d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:21.654391 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce786f-b2ca-4fd8-98bb-13a6c043c02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:21.654684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce786f-b2ca-4fd8-98bb-13a6c043c02d admin admin] PATH_INFO: `/v3/projects/e8d5838572ec4b17a252b6386ca50dc1` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:21.657134 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce786f-b2ca-4fd8-98bb-13a6c043c02d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8d5838572ec4b17a252b6386ca50dc1)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:21.659730 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce786f-b2ca-4fd8-98bb-13a6c043c02d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:21.749206 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-53ce786f-b2ca-4fd8-98bb-13a6c043c02d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8d5838572ec4b17a252b6386ca50dc1', 'request_id': 'req-53ce786f-b2ca-4fd8-98bb-13a6c043c02d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:21.765551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-53ce786f-b2ca-4fd8-98bb-13a6c043c02d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8d5838572ec4b17a252b6386ca50dc1', 'request_id': 'req-53ce786f-b2ca-4fd8-98bb-13a6c043c02d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:21.777027 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-53ce786f-b2ca-4fd8-98bb-13a6c043c02d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8d5838572ec4b17a252b6386ca50dc1', 'request_id': 'req-53ce786f-b2ca-4fd8-98bb-13a6c043c02d'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:21.803663 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-53ce786f-b2ca-4fd8-98bb-13a6c043c02d admin admin] The token cache is being invalidate because project e8d5838572ec4b17a252b6386ca50dc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:33:21.803960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-53ce786f-b2ca-4fd8-98bb-13a6c043c02d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:21.805982 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-53ce786f-b2ca-4fd8-98bb-13a6c043c02d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:21.817773 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1123/2261] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:33:21 2022] DELETE /identity/v3/projects/e8d5838572ec4b17a252b6386ca50dc1 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:21.822194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7310a-640a-47c0-8b92-e9d54c4569fc None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:21.823030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7310a-640a-47c0-8b92-e9d54c4569fc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:21.823743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7310a-640a-47c0-8b92-e9d54c4569fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:21.866903 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c1f7310a-640a-47c0-8b92-e9d54c4569fc None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:21.869107 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c1f7310a-640a-47c0-8b92-e9d54c4569fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:21.995126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1139/2262] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:22.008772 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b056be8-ec72-4d9e-98a5-a3390ced297e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:22.009069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b056be8-ec72-4d9e-98a5-a3390ced297e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:22.010196 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b056be8-ec72-4d9e-98a5-a3390ced297e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:22.054170 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5b056be8-ec72-4d9e-98a5-a3390ced297e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:22.057834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5b056be8-ec72-4d9e-98a5-a3390ced297e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:22.090444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1124/2263] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:33:22.095529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7310a-640a-47c0-8b92-e9d54c4569fc None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:22.112982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7310a-640a-47c0-8b92-e9d54c4569fc None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:22.130375 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335c256-a507-4dc1-bfc1-6ab82aa141f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:22.131825 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335c256-a507-4dc1-bfc1-6ab82aa141f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:22.131825 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335c256-a507-4dc1-bfc1-6ab82aa141f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:22.131825 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335c256-a507-4dc1-bfc1-6ab82aa141f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:22.132578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335c256-a507-4dc1-bfc1-6ab82aa141f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:33:22.132876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335c256-a507-4dc1-bfc1-6ab82aa141f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:22.142783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b056be8-ec72-4d9e-98a5-a3390ced297e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:22.143947 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335c256-a507-4dc1-bfc1-6ab82aa141f8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:22.144511 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5b056be8-ec72-4d9e-98a5-a3390ced297e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:22.161016 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b056be8-ec72-4d9e-98a5-a3390ced297e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:22.168810 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1140/2264] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:33:22.176113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335c256-a507-4dc1-bfc1-6ab82aa141f8 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:22.179070 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1c28a-3e28-4654-b8c0-47057a80157a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:22.180026 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1c28a-3e28-4654-b8c0-47057a80157a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:22.180588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1c28a-3e28-4654-b8c0-47057a80157a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:22.180916 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1c28a-3e28-4654-b8c0-47057a80157a admin admin] PATH_INFO: `/v3/projects/9a72e63c1c1c4b1dbc0bf355c1ff6d63` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:22.182396 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1c28a-3e28-4654-b8c0-47057a80157a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a72e63c1c1c4b1dbc0bf355c1ff6d63)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:22.184462 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1c28a-3e28-4654-b8c0-47057a80157a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:22.204678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335c256-a507-4dc1-bfc1-6ab82aa141f8 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:22.213977 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519939a-40db-4d37-aa9d-2e8c3f817e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:22.215028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519939a-40db-4d37-aa9d-2e8c3f817e9d admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:22.215506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519939a-40db-4d37-aa9d-2e8c3f817e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:22.215981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519939a-40db-4d37-aa9d-2e8c3f817e9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:22.217202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519939a-40db-4d37-aa9d-2e8c3f817e9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:22.220973 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519939a-40db-4d37-aa9d-2e8c3f817e9d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:22.281826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1141/2265] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:33:22 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:33:22.287425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a1f1c28a-3e28-4654-b8c0-47057a80157a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a72e63c1c1c4b1dbc0bf355c1ff6d63', 'request_id': 'req-a1f1c28a-3e28-4654-b8c0-47057a80157a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:22.287810 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519939a-40db-4d37-aa9d-2e8c3f817e9d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:22.300364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a1f1c28a-3e28-4654-b8c0-47057a80157a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a72e63c1c1c4b1dbc0bf355c1ff6d63', 'request_id': 'req-a1f1c28a-3e28-4654-b8c0-47057a80157a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:22.312893 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a1f1c28a-3e28-4654-b8c0-47057a80157a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a72e63c1c1c4b1dbc0bf355c1ff6d63', 'request_id': 'req-a1f1c28a-3e28-4654-b8c0-47057a80157a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:22.342815 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a1f1c28a-3e28-4654-b8c0-47057a80157a admin admin] The token cache is being invalidate because project 9a72e63c1c1c4b1dbc0bf355c1ff6d63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:33:22.343485 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a1f1c28a-3e28-4654-b8c0-47057a80157a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:22.345015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-a1f1c28a-3e28-4654-b8c0-47057a80157a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:22.349478 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1125/2266] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:33:22 2022] DELETE /identity/v3/projects/9a72e63c1c1c4b1dbc0bf355c1ff6d63 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:22.357399 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e045b54-60c8-48cf-b323-2399ebbf8993 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:22.357895 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e045b54-60c8-48cf-b323-2399ebbf8993 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:22.358394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e045b54-60c8-48cf-b323-2399ebbf8993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:22.366431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:22.389207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519939a-40db-4d37-aa9d-2e8c3f817e9d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:22.390493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5e045b54-60c8-48cf-b323-2399ebbf8993 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:22.392400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5e045b54-60c8-48cf-b323-2399ebbf8993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:22.397900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835a8de-4303-42fa-84ef-8f9c1ede2d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:22.399077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835a8de-4303-42fa-84ef-8f9c1ede2d15 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:22.399542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835a8de-4303-42fa-84ef-8f9c1ede2d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:22.399964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835a8de-4303-42fa-84ef-8f9c1ede2d15 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:22.401176 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835a8de-4303-42fa-84ef-8f9c1ede2d15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:22.404783 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835a8de-4303-42fa-84ef-8f9c1ede2d15 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:22.406490 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1126/2267] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:22.420193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ad712-cb0c-499e-832e-0f3c41db8968 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:22.420572 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ad712-cb0c-499e-832e-0f3c41db8968 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:22.421045 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ad712-cb0c-499e-832e-0f3c41db8968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:22.440987 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1142/2268] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:33:22 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:33:22.442157 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835a8de-4303-42fa-84ef-8f9c1ede2d15 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:22.444827 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3835a8de-4303-42fa-84ef-8f9c1ede2d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:22.446375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c36ad712-cb0c-499e-832e-0f3c41db8968 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:22.447610 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c36ad712-cb0c-499e-832e-0f3c41db8968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:22.457919 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835a8de-4303-42fa-84ef-8f9c1ede2d15 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:22.458359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1127/2269] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:22.463110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ad712-cb0c-499e-832e-0f3c41db8968 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:22.464059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a2648-b2c0-46ef-9083-c7574b54411e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:22.465316 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a2648-b2c0-46ef-9083-c7574b54411e admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:22.465857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a2648-b2c0-46ef-9083-c7574b54411e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:22.466379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a2648-b2c0-46ef-9083-c7574b54411e admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:22.467848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a2648-b2c0-46ef-9083-c7574b54411e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:22.470468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a2648-b2c0-46ef-9083-c7574b54411e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:22.474345 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ad712-cb0c-499e-832e-0f3c41db8968 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:22.481600 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88a01e-1002-4d6f-be82-a63d99a68f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:22.482725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88a01e-1002-4d6f-be82-a63d99a68f7b admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:22.483167 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88a01e-1002-4d6f-be82-a63d99a68f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:22.483601 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88a01e-1002-4d6f-be82-a63d99a68f7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:22.484505 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88a01e-1002-4d6f-be82-a63d99a68f7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:33:22.484980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88a01e-1002-4d6f-be82-a63d99a68f7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:22.486851 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88a01e-1002-4d6f-be82-a63d99a68f7b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:22.487949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1143/2270] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:22.492872 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a2648-b2c0-46ef-9083-c7574b54411e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:22.505491 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1128/2271] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:33:22.508830 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88a01e-1002-4d6f-be82-a63d99a68f7b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:22.513035 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a2648-b2c0-46ef-9083-c7574b54411e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:22.522108 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88a01e-1002-4d6f-be82-a63d99a68f7b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:22.522713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16397e30-6bdd-4f99-ae5f-251b8b7e1835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:22.523552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16397e30-6bdd-4f99-ae5f-251b8b7e1835 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:22.523778 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16397e30-6bdd-4f99-ae5f-251b8b7e1835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:22.524060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16397e30-6bdd-4f99-ae5f-251b8b7e1835 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:22.524835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16397e30-6bdd-4f99-ae5f-251b8b7e1835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:22.526672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16397e30-6bdd-4f99-ae5f-251b8b7e1835 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:22.528332 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccc981-cf0b-4726-8de3-628a5e96a0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:22.529904 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccc981-cf0b-4726-8de3-628a5e96a0a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:22.530351 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccc981-cf0b-4726-8de3-628a5e96a0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:22.530738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccc981-cf0b-4726-8de3-628a5e96a0a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:22.531572 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccc981-cf0b-4726-8de3-628a5e96a0a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:22.535357 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccc981-cf0b-4726-8de3-628a5e96a0a0 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:22.541272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1144/2272] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:33:22.545886 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16397e30-6bdd-4f99-ae5f-251b8b7e1835 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:22.560409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:22.563282 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16397e30-6bdd-4f99-ae5f-251b8b7e1835 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:22.572768 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e12a14-6f4d-41b7-9214-7c91eb2bc646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:22.574112 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e12a14-6f4d-41b7-9214-7c91eb2bc646 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:22.574710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e12a14-6f4d-41b7-9214-7c91eb2bc646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:22.575468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e12a14-6f4d-41b7-9214-7c91eb2bc646 admin admin] PATH_INFO: `/v3/projects/f9febbdff58a49bbb56b0e64aaa851c7/users/07ff7a74b1424b9288de71926c97f544/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:22.586659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1129/2273] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:33:22 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:33:22.591055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccc981-cf0b-4726-8de3-628a5e96a0a0 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:22.599695 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e12a14-6f4d-41b7-9214-7c91eb2bc646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9febbdff58a49bbb56b0e64aaa851c7, user_id=07ff7a74b1424b9288de71926c97f544, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:22.606919 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e12a14-6f4d-41b7-9214-7c91eb2bc646 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:22.635235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1145/2274] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:33:22 2022] PUT /identity/v3/projects/f9febbdff58a49bbb56b0e64aaa851c7/users/07ff7a74b1424b9288de71926c97f544/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:22.647933 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd76c73-a3ad-4d90-a031-5328f829a579 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:22.648211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd76c73-a3ad-4d90-a031-5328f829a579 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:22.649344 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd76c73-a3ad-4d90-a031-5328f829a579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:22.687419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9bd76c73-a3ad-4d90-a031-5328f829a579 None None] MFA Rules not processed for user `07ff7a74b1424b9288de71926c97f544`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:22.689567 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9bd76c73-a3ad-4d90-a031-5328f829a579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:22.716784 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccc981-cf0b-4726-8de3-628a5e96a0a0 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:22.817823 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03874046-f18a-469f-8875-9f21bf65b063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:22.818591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03874046-f18a-469f-8875-9f21bf65b063 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:22.818992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03874046-f18a-469f-8875-9f21bf65b063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:22.819384 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03874046-f18a-469f-8875-9f21bf65b063 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:22.820240 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03874046-f18a-469f-8875-9f21bf65b063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:22.828672 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03874046-f18a-469f-8875-9f21bf65b063 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:22.845802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1146/2275] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:33:22.851472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd76c73-a3ad-4d90-a031-5328f829a579 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:22.865030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd76c73-a3ad-4d90-a031-5328f829a579 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:22.870470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1130/2276] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:33:22 2022] POST /identity/v3/users => generated 292 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:33:22.875190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03874046-f18a-469f-8875-9f21bf65b063 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:22.876026 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac0691-082e-4fb5-b046-cd9d6ddbf0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:22.876780 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac0691-082e-4fb5-b046-cd9d6ddbf0cc admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:22.877061 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac0691-082e-4fb5-b046-cd9d6ddbf0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:22.877383 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac0691-082e-4fb5-b046-cd9d6ddbf0cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:22.881529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac0691-082e-4fb5-b046-cd9d6ddbf0cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:22.883667 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac0691-082e-4fb5-b046-cd9d6ddbf0cc admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:22.889608 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03874046-f18a-469f-8875-9f21bf65b063 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:22.895878 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765db15a-0870-441b-8a7b-2afd8ffb86d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:22.896873 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765db15a-0870-441b-8a7b-2afd8ffb86d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:22.897164 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765db15a-0870-441b-8a7b-2afd8ffb86d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:22.897479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765db15a-0870-441b-8a7b-2afd8ffb86d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:22.898387 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765db15a-0870-441b-8a7b-2afd8ffb86d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:22.901365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765db15a-0870-441b-8a7b-2afd8ffb86d3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:22.914922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1131/2277] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:22.917327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1147/2278] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:33:22 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:33:22.919315 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765db15a-0870-441b-8a7b-2afd8ffb86d3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:22.919469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac0691-082e-4fb5-b046-cd9d6ddbf0cc admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:22.932415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765db15a-0870-441b-8a7b-2afd8ffb86d3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:23.019214 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:23.023615 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc37a3-d107-4dd2-9e40-bb7276810b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:23.024843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc37a3-d107-4dd2-9e40-bb7276810b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:23.024843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc37a3-d107-4dd2-9e40-bb7276810b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:23.024843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc37a3-d107-4dd2-9e40-bb7276810b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:23.025629 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc37a3-d107-4dd2-9e40-bb7276810b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:23.027376 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc37a3-d107-4dd2-9e40-bb7276810b44 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:23.027873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac0691-082e-4fb5-b046-cd9d6ddbf0cc admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:23.037169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a467b8-2146-4cdb-88e3-8ab6f8c3c641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:23.037878 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a467b8-2146-4cdb-88e3-8ab6f8c3c641 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:23.038161 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a467b8-2146-4cdb-88e3-8ab6f8c3c641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:23.038473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a467b8-2146-4cdb-88e3-8ab6f8c3c641 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:23.039145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a467b8-2146-4cdb-88e3-8ab6f8c3c641 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:23.041319 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a467b8-2146-4cdb-88e3-8ab6f8c3c641 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:23.045544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1132/2279] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:23.048442 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc37a3-d107-4dd2-9e40-bb7276810b44 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:23.063030 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc37a3-d107-4dd2-9e40-bb7276810b44 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:23.073777 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1148/2280] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:33:22 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:33:23.073904 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf487a20-af37-42a1-92c1-5e799e25214a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:23.073904 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf487a20-af37-42a1-92c1-5e799e25214a admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:23.073904 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf487a20-af37-42a1-92c1-5e799e25214a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:23.073904 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf487a20-af37-42a1-92c1-5e799e25214a admin admin] PATH_INFO: `/v3/projects/14bd96bd91c44b888074b760e9834814/users/d4008c6bb57b4a91b2761fc87a1e9d2f/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:23.076140 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a467b8-2146-4cdb-88e3-8ab6f8c3c641 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:23.093045 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a467b8-2146-4cdb-88e3-8ab6f8c3c641 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:23.094239 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:23.098077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf487a20-af37-42a1-92c1-5e799e25214a admin admin] RBAC: Authorizing `identity:create_grant(project_id=14bd96bd91c44b888074b760e9834814, user_id=d4008c6bb57b4a91b2761fc87a1e9d2f, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:23.098324 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035611f0-03d7-4f40-96b2-20f6cf9c66fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:23.099013 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035611f0-03d7-4f40-96b2-20f6cf9c66fa admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:23.099302 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035611f0-03d7-4f40-96b2-20f6cf9c66fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:23.099581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035611f0-03d7-4f40-96b2-20f6cf9c66fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:23.100319 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035611f0-03d7-4f40-96b2-20f6cf9c66fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:23.100822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf487a20-af37-42a1-92c1-5e799e25214a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:23.101995 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035611f0-03d7-4f40-96b2-20f6cf9c66fa admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:23.117987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1133/2281] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:33:23 2022] PUT /identity/v3/projects/14bd96bd91c44b888074b760e9834814/users/d4008c6bb57b4a91b2761fc87a1e9d2f/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:33:23.121536 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1149/2282] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:23.122864 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf487a20-af37-42a1-92c1-5e799e25214a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:23.126560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d0408-9f23-44d1-8aa7-e17e3f5e2c70 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:23.127030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d0408-9f23-44d1-8aa7-e17e3f5e2c70 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:23.128042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d0408-9f23-44d1-8aa7-e17e3f5e2c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:23.150166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-938d0408-9f23-44d1-8aa7-e17e3f5e2c70 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:23.151392 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-938d0408-9f23-44d1-8aa7-e17e3f5e2c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:23.247247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1150/2283] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:23.259658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf487a20-af37-42a1-92c1-5e799e25214a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:23.264823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d0408-9f23-44d1-8aa7-e17e3f5e2c70 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:23.266853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-938d0408-9f23-44d1-8aa7-e17e3f5e2c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:23.269385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd9ead-b4d8-4a71-a695-43b32c354557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:23.270453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd9ead-b4d8-4a71-a695-43b32c354557 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:23.270950 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd9ead-b4d8-4a71-a695-43b32c354557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:23.271481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd9ead-b4d8-4a71-a695-43b32c354557 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:23.272841 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd9ead-b4d8-4a71-a695-43b32c354557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:23.275614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd9ead-b4d8-4a71-a695-43b32c354557 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:23.300913 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1134/2284] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:23.304042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd9ead-b4d8-4a71-a695-43b32c354557 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:23.326869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd9ead-b4d8-4a71-a695-43b32c354557 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:23.342557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285d0ec-a42c-4573-b13c-52a9491dc79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:23.345789 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285d0ec-a42c-4573-b13c-52a9491dc79f admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:23.348138 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285d0ec-a42c-4573-b13c-52a9491dc79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:23.348519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285d0ec-a42c-4573-b13c-52a9491dc79f admin admin] PATH_INFO: `/v3/projects/a43ddb55909c48c3894a1731d01c7576/users/8146b6bbd2744671a9f59c0b6c8ce287/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:23.382243 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d0408-9f23-44d1-8aa7-e17e3f5e2c70 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:23.383932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285d0ec-a42c-4573-b13c-52a9491dc79f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a43ddb55909c48c3894a1731d01c7576, user_id=8146b6bbd2744671a9f59c0b6c8ce287, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:23.386667 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52d699-82d3-491e-af7a-423843a45454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:23.387337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52d699-82d3-491e-af7a-423843a45454 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:23.387627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52d699-82d3-491e-af7a-423843a45454 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:23.387877 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52d699-82d3-491e-af7a-423843a45454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:23.390848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285d0ec-a42c-4573-b13c-52a9491dc79f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:23.400617 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52d699-82d3-491e-af7a-423843a45454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:23.403197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52d699-82d3-491e-af7a-423843a45454 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:23.420337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1151/2285] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:33:23.429365 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1135/2286] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:33:23 2022] PUT /identity/v3/projects/a43ddb55909c48c3894a1731d01c7576/users/8146b6bbd2744671a9f59c0b6c8ce287/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:23.433009 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a645e90-a85e-4285-b5d0-8e99295cd8f0 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:23.433342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a645e90-a85e-4285-b5d0-8e99295cd8f0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:23.433634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a645e90-a85e-4285-b5d0-8e99295cd8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:23.464171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3a645e90-a85e-4285-b5d0-8e99295cd8f0 None None] MFA Rules not processed for user `8146b6bbd2744671a9f59c0b6c8ce287`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:23.465635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3a645e90-a85e-4285-b5d0-8e99295cd8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:23.542994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1152/2287] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:23.547642 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285d0ec-a42c-4573-b13c-52a9491dc79f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:23.628572 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285d0ec-a42c-4573-b13c-52a9491dc79f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:23.634954 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb040de9-a557-48d2-b5b0-cde6b68d1c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:23.635778 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb040de9-a557-48d2-b5b0-cde6b68d1c08 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:23.636077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb040de9-a557-48d2-b5b0-cde6b68d1c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:23.636574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb040de9-a557-48d2-b5b0-cde6b68d1c08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:23.637468 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb040de9-a557-48d2-b5b0-cde6b68d1c08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:23.639626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb040de9-a557-48d2-b5b0-cde6b68d1c08 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:23.682048 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1136/2288] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:33:23 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:33:23.686323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a645e90-a85e-4285-b5d0-8e99295cd8f0 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:23.698856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a645e90-a85e-4285-b5d0-8e99295cd8f0 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:23.705411 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:23.784743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7356955-7560-4100-b105-83333888a217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:23.785618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7356955-7560-4100-b105-83333888a217 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:23.785902 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7356955-7560-4100-b105-83333888a217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:23.786223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7356955-7560-4100-b105-83333888a217 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:23.787000 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7356955-7560-4100-b105-83333888a217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:23.789139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7356955-7560-4100-b105-83333888a217 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:23.820065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1153/2289] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:33:23 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:33:23.823498 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb040de9-a557-48d2-b5b0-cde6b68d1c08 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:23.851663 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb040de9-a557-48d2-b5b0-cde6b68d1c08 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:23.863701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd9daa-3024-462c-b0d6-23b0de5ebd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:23.864739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd9daa-3024-462c-b0d6-23b0de5ebd41 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:23.869340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd9daa-3024-462c-b0d6-23b0de5ebd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:23.869340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd9daa-3024-462c-b0d6-23b0de5ebd41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:23.869340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd9daa-3024-462c-b0d6-23b0de5ebd41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:23.871861 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd9daa-3024-462c-b0d6-23b0de5ebd41 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:23.898505 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1137/2290] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:23.899231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7356955-7560-4100-b105-83333888a217 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:23.914627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7356955-7560-4100-b105-83333888a217 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:23.919814 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bade05c-ab4d-4bce-be53-0265cb47b430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:23.920729 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bade05c-ab4d-4bce-be53-0265cb47b430 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:23.921015 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bade05c-ab4d-4bce-be53-0265cb47b430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:23.921495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bade05c-ab4d-4bce-be53-0265cb47b430 admin admin] PATH_INFO: `/v3/projects/ccaa6b270e6e492cb7bd2322285fbb1e/users/d7fc99e610894930b45deea82fba3df7/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:23.941391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bade05c-ab4d-4bce-be53-0265cb47b430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccaa6b270e6e492cb7bd2322285fbb1e, user_id=d7fc99e610894930b45deea82fba3df7, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:23.944153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bade05c-ab4d-4bce-be53-0265cb47b430 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:23.960238 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1154/2291] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:33:23 2022] PUT /identity/v3/projects/ccaa6b270e6e492cb7bd2322285fbb1e/users/d7fc99e610894930b45deea82fba3df7/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:23.965206 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd9daa-3024-462c-b0d6-23b0de5ebd41 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:23.980841 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd9daa-3024-462c-b0d6-23b0de5ebd41 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:24.024206 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:24.063619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9a22f-7457-476f-8a4c-287b07bfc9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:24.064314 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9a22f-7457-476f-8a4c-287b07bfc9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:24.064594 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9a22f-7457-476f-8a4c-287b07bfc9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:24.065693 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9a22f-7457-476f-8a4c-287b07bfc9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:24.066380 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9a22f-7457-476f-8a4c-287b07bfc9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:24.068046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9a22f-7457-476f-8a4c-287b07bfc9f2 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:24.083081 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1138/2292] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:24.088519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bade05c-ab4d-4bce-be53-0265cb47b430 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:24.101994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bade05c-ab4d-4bce-be53-0265cb47b430 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:24.107266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22baf3c7-0aaa-4db7-9a1d-f6304c69c544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:24.108711 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22baf3c7-0aaa-4db7-9a1d-f6304c69c544 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:24.109184 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22baf3c7-0aaa-4db7-9a1d-f6304c69c544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:24.109628 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22baf3c7-0aaa-4db7-9a1d-f6304c69c544 admin admin] PATH_INFO: `/v3/projects/ccaa6b270e6e492cb7bd2322285fbb1e/users/d7fc99e610894930b45deea82fba3df7/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:24.112454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22baf3c7-0aaa-4db7-9a1d-f6304c69c544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccaa6b270e6e492cb7bd2322285fbb1e, user_id=d7fc99e610894930b45deea82fba3df7, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:24.115527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22baf3c7-0aaa-4db7-9a1d-f6304c69c544 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:24.135557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1155/2293] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:33:24 2022] PUT /identity/v3/projects/ccaa6b270e6e492cb7bd2322285fbb1e/users/d7fc99e610894930b45deea82fba3df7/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:24.150418 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3078aef-ed5c-4d67-b2c5-a248acdfb059 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:24.150875 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3078aef-ed5c-4d67-b2c5-a248acdfb059 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:24.151854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3078aef-ed5c-4d67-b2c5-a248acdfb059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:24.188274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b3078aef-ed5c-4d67-b2c5-a248acdfb059 None None] MFA Rules not processed for user `d7fc99e610894930b45deea82fba3df7`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:24.189547 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b3078aef-ed5c-4d67-b2c5-a248acdfb059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:24.274143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1139/2294] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:33:24.286329 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22baf3c7-0aaa-4db7-9a1d-f6304c69c544 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:24.288376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-22baf3c7-0aaa-4db7-9a1d-f6304c69c544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:24.370281 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22baf3c7-0aaa-4db7-9a1d-f6304c69c544 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:24.373868 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69481f-c5f3-4266-b19b-3f90f16a96e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:24.374550 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69481f-c5f3-4266-b19b-3f90f16a96e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:24.374836 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69481f-c5f3-4266-b19b-3f90f16a96e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:24.375117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69481f-c5f3-4266-b19b-3f90f16a96e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:24.526297 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a69481f-c5f3-4266-b19b-3f90f16a96e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:24.528130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a69481f-c5f3-4266-b19b-3f90f16a96e9 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:24.544217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1156/2295] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:33:26.314195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3078aef-ed5c-4d67-b2c5-a248acdfb059 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:26.336864 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3078aef-ed5c-4d67-b2c5-a248acdfb059 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:26.341115 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d117df-9910-4d1b-b303-945216cfe500 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:26.341810 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d117df-9910-4d1b-b303-945216cfe500 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:26.342755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d117df-9910-4d1b-b303-945216cfe500 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:26.342755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d117df-9910-4d1b-b303-945216cfe500 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:26.411294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d117df-9910-4d1b-b303-945216cfe500 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:26.412769 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d117df-9910-4d1b-b303-945216cfe500 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:26.423811 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1140/2296] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:33:27.465870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69481f-c5f3-4266-b19b-3f90f16a96e9 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:27.486663 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69481f-c5f3-4266-b19b-3f90f16a96e9 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:27.498800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f77bb5-1008-4d73-98f2-754ab0c1cd2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:27.499854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f77bb5-1008-4d73-98f2-754ab0c1cd2e service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:27.500390 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f77bb5-1008-4d73-98f2-754ab0c1cd2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:27.500902 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f77bb5-1008-4d73-98f2-754ab0c1cd2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:27.528688 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f77bb5-1008-4d73-98f2-754ab0c1cd2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:27.530217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f77bb5-1008-4d73-98f2-754ab0c1cd2e service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:27.555799 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1157/2297] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:33:30.329052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409727d7-964c-4cb1-a85a-e9c29494b6e1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:30.329550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409727d7-964c-4cb1-a85a-e9c29494b6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:30.329651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409727d7-964c-4cb1-a85a-e9c29494b6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:30.384165 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-409727d7-964c-4cb1-a85a-e9c29494b6e1 None None] MFA Rules not processed for user `d4008c6bb57b4a91b2761fc87a1e9d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:30.386937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-409727d7-964c-4cb1-a85a-e9c29494b6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:30.514114 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:30.562036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1141/2298] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:30.562925 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f77bb5-1008-4d73-98f2-754ab0c1cd2e service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:30.568599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:30.667088 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f77bb5-1008-4d73-98f2-754ab0c1cd2e service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:30.672846 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c59d0-61ee-495a-ba74-44bcf2dfefe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:30.673893 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c59d0-61ee-495a-ba74-44bcf2dfefe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:30.674935 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c59d0-61ee-495a-ba74-44bcf2dfefe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:30.675488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c59d0-61ee-495a-ba74-44bcf2dfefe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:30.676413 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c59d0-61ee-495a-ba74-44bcf2dfefe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:30.678520 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c59d0-61ee-495a-ba74-44bcf2dfefe8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:30.725651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1158/2299] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:33:30 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:33:30.728905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409727d7-964c-4cb1-a85a-e9c29494b6e1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:30.748558 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409727d7-964c-4cb1-a85a-e9c29494b6e1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:30.835188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b4fd0-6459-446d-a9c8-29f22b746e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:30.835889 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b4fd0-6459-446d-a9c8-29f22b746e03 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:30.836179 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b4fd0-6459-446d-a9c8-29f22b746e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:30.836462 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b4fd0-6459-446d-a9c8-29f22b746e03 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:30.837173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b4fd0-6459-446d-a9c8-29f22b746e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:30.842934 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b4fd0-6459-446d-a9c8-29f22b746e03 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:30.876644 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1142/2300] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:33:30 2022] POST /identity/v3/users => generated 299 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:33:30.880735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c59d0-61ee-495a-ba74-44bcf2dfefe8 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:30.914048 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c59d0-61ee-495a-ba74-44bcf2dfefe8 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:30.935339 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aacb91-9055-4f1f-9225-97d87f4a0dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:30.937046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aacb91-9055-4f1f-9225-97d87f4a0dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:30.937625 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aacb91-9055-4f1f-9225-97d87f4a0dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:30.938167 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aacb91-9055-4f1f-9225-97d87f4a0dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:30.939595 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aacb91-9055-4f1f-9225-97d87f4a0dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:30.943067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aacb91-9055-4f1f-9225-97d87f4a0dd9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:30.969214 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1159/2301] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:30.973166 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b4fd0-6459-446d-a9c8-29f22b746e03 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:30.996705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b4fd0-6459-446d-a9c8-29f22b746e03 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:31.018722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3a489-0836-4443-a972-fcfb2295884e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:31.021772 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3a489-0836-4443-a972-fcfb2295884e admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:31.022071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3a489-0836-4443-a972-fcfb2295884e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:31.022354 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3a489-0836-4443-a972-fcfb2295884e admin admin] PATH_INFO: `/v3/projects/f830b7e66df64a10aab07af6820ef125/users/75ae2c185a764de1864f87a1d109791d/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:31.051535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3a489-0836-4443-a972-fcfb2295884e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f830b7e66df64a10aab07af6820ef125, user_id=75ae2c185a764de1864f87a1d109791d, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:31.054594 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3a489-0836-4443-a972-fcfb2295884e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:31.072958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1143/2302] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:33:30 2022] PUT /identity/v3/projects/f830b7e66df64a10aab07af6820ef125/users/75ae2c185a764de1864f87a1d109791d/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:31.077423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aacb91-9055-4f1f-9225-97d87f4a0dd9 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:31.118911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aacb91-9055-4f1f-9225-97d87f4a0dd9 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:31.209478 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affd1db-6578-4cf1-99bb-a19e1d0887a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:31.210186 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affd1db-6578-4cf1-99bb-a19e1d0887a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:31.210474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affd1db-6578-4cf1-99bb-a19e1d0887a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:31.210755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affd1db-6578-4cf1-99bb-a19e1d0887a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:31.211509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5affd1db-6578-4cf1-99bb-a19e1d0887a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:31.214181 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5affd1db-6578-4cf1-99bb-a19e1d0887a7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:31.229389 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1160/2303] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:31.230385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3a489-0836-4443-a972-fcfb2295884e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:31.242876 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3a489-0836-4443-a972-fcfb2295884e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:31.247914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac507d52-a9b6-426e-9da9-596dc426d314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:31.248863 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac507d52-a9b6-426e-9da9-596dc426d314 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:31.249149 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac507d52-a9b6-426e-9da9-596dc426d314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:31.249583 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac507d52-a9b6-426e-9da9-596dc426d314 admin admin] PATH_INFO: `/v3/projects/f830b7e66df64a10aab07af6820ef125/users/75ae2c185a764de1864f87a1d109791d/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:31.251654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac507d52-a9b6-426e-9da9-596dc426d314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f830b7e66df64a10aab07af6820ef125, user_id=75ae2c185a764de1864f87a1d109791d, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:31.255185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac507d52-a9b6-426e-9da9-596dc426d314 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:31.282303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1144/2304] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:33:31 2022] PUT /identity/v3/projects/f830b7e66df64a10aab07af6820ef125/users/75ae2c185a764de1864f87a1d109791d/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:33:32.240515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affd1db-6578-4cf1-99bb-a19e1d0887a7 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:32.281273 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affd1db-6578-4cf1-99bb-a19e1d0887a7 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:32.591923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16985558-de87-414d-b44d-8164bff28822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:32.591923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16985558-de87-414d-b44d-8164bff28822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:32.591923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16985558-de87-414d-b44d-8164bff28822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:32.591923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16985558-de87-414d-b44d-8164bff28822 admin admin] PATH_INFO: `/v3/users/07ff7a74b1424b9288de71926c97f544` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:32.617838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16985558-de87-414d-b44d-8164bff28822 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07ff7a74b1424b9288de71926c97f544)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:32.623622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16985558-de87-414d-b44d-8164bff28822 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:32.707071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:32.851248 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-16985558-de87-414d-b44d-8164bff28822 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07ff7a74b1424b9288de71926c97f544'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:32.854578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-16985558-de87-414d-b44d-8164bff28822 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07ff7a74b1424b9288de71926c97f544'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:32.898282 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d243bd8-2937-4052-ba73-1be5b11b7577 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:32.899301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-16985558-de87-414d-b44d-8164bff28822 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07ff7a74b1424b9288de71926c97f544'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:32.899607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d243bd8-2937-4052-ba73-1be5b11b7577 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:32.899887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d243bd8-2937-4052-ba73-1be5b11b7577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:32.900310 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-16985558-de87-414d-b44d-8164bff28822 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07ff7a74b1424b9288de71926c97f544'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:32.922274 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-16985558-de87-414d-b44d-8164bff28822 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07ff7a74b1424b9288de71926c97f544'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:32.945361 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1161/2305] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:33:32 2022] DELETE /identity/v3/users/07ff7a74b1424b9288de71926c97f544 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:32.952940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7c318-c688-479e-8d33-76a17ed769ab None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:32.953685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7c318-c688-479e-8d33-76a17ed769ab None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:32.954290 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7c318-c688-479e-8d33-76a17ed769ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:33.077888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4d243bd8-2937-4052-ba73-1be5b11b7577 None None] MFA Rules not processed for user `aee77a3ed91445ab8b7f58e7f8bc348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:33.079202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4d243bd8-2937-4052-ba73-1be5b11b7577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:33.104598 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-c6b7c318-c688-479e-8d33-76a17ed769ab None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:33.116022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c6b7c318-c688-479e-8d33-76a17ed769ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:33.189092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:33.226034 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1145/2306] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 06:33:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:33:33.265074 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d243bd8-2937-4052-ba73-1be5b11b7577 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:33.267386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4d243bd8-2937-4052-ba73-1be5b11b7577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:33.339837 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1162/2307] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:33:33.382871 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7c318-c688-479e-8d33-76a17ed769ab None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:33.387138 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c6b7c318-c688-479e-8d33-76a17ed769ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:33.540049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d243bd8-2937-4052-ba73-1be5b11b7577 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:33.556470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad279a9-ea5d-452c-9919-5da724ac1225 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc205d053d54506bb6c9227e4a1b044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:33.557667 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad279a9-ea5d-452c-9919-5da724ac1225 service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:33.558303 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad279a9-ea5d-452c-9919-5da724ac1225 service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:33.558698 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad279a9-ea5d-452c-9919-5da724ac1225 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:33.574761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad279a9-ea5d-452c-9919-5da724ac1225 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:33.577635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad279a9-ea5d-452c-9919-5da724ac1225 service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:33.587491 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7c318-c688-479e-8d33-76a17ed769ab None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:33.611278 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171656fd-df6a-40ba-ab51-6ab17a63b31f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:33.615548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1146/2308] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 06:33:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 06:33:33.617447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171656fd-df6a-40ba-ab51-6ab17a63b31f service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:33.617765 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171656fd-df6a-40ba-ab51-6ab17a63b31f service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:33.618052 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171656fd-df6a-40ba-ab51-6ab17a63b31f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:33.715890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171656fd-df6a-40ba-ab51-6ab17a63b31f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:33.730723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171656fd-df6a-40ba-ab51-6ab17a63b31f service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:33.758912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1163/2309] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:33:35.170507 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6493943-723c-47ff-b622-b1298ef0a2b1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:35.173059 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6493943-723c-47ff-b622-b1298ef0a2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:35.174344 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6493943-723c-47ff-b622-b1298ef0a2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:35.230738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f6493943-723c-47ff-b622-b1298ef0a2b1 None None] MFA Rules not processed for user `aee77a3ed91445ab8b7f58e7f8bc348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:35.233120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f6493943-723c-47ff-b622-b1298ef0a2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:35.256091 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1147/2310] 10.222.0.65 () {58 vars in 967 bytes} [Thu May 19 06:33:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:35.262964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171656fd-df6a-40ba-ab51-6ab17a63b31f service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:35.293324 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171656fd-df6a-40ba-ab51-6ab17a63b31f service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:35.305910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c495e18-37e8-422e-a0d4-78c4b6296c72 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc205d053d54506bb6c9227e4a1b044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:35.307998 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c495e18-37e8-422e-a0d4-78c4b6296c72 service swift] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:35.308793 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c495e18-37e8-422e-a0d4-78c4b6296c72 service swift] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:35.309475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c495e18-37e8-422e-a0d4-78c4b6296c72 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:35.362136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c495e18-37e8-422e-a0d4-78c4b6296c72 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:35.368797 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c495e18-37e8-422e-a0d4-78c4b6296c72 service swift] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:35.414722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1164/2311] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 06:33:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 06:33:35.624668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6493943-723c-47ff-b622-b1298ef0a2b1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:35.629615 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f6493943-723c-47ff-b622-b1298ef0a2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:35.677667 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6493943-723c-47ff-b622-b1298ef0a2b1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:35.690025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d46b4-7cb4-4955-9f3d-55a69ac819ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:35.691668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d46b4-7cb4-4955-9f3d-55a69ac819ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:35.692173 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d46b4-7cb4-4955-9f3d-55a69ac819ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:35.692791 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d46b4-7cb4-4955-9f3d-55a69ac819ec admin admin] PATH_INFO: `/v3/projects/f9febbdff58a49bbb56b0e64aaa851c7` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:35.700669 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d46b4-7cb4-4955-9f3d-55a69ac819ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9febbdff58a49bbb56b0e64aaa851c7)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:35.704355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d46b4-7cb4-4955-9f3d-55a69ac819ec admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:35.936400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c86d46b4-7cb4-4955-9f3d-55a69ac819ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9febbdff58a49bbb56b0e64aaa851c7', 'request_id': 'req-c86d46b4-7cb4-4955-9f3d-55a69ac819ec'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:35.972949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c86d46b4-7cb4-4955-9f3d-55a69ac819ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9febbdff58a49bbb56b0e64aaa851c7', 'request_id': 'req-c86d46b4-7cb4-4955-9f3d-55a69ac819ec'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:35.993319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c86d46b4-7cb4-4955-9f3d-55a69ac819ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9febbdff58a49bbb56b0e64aaa851c7', 'request_id': 'req-c86d46b4-7cb4-4955-9f3d-55a69ac819ec'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:36.045599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c86d46b4-7cb4-4955-9f3d-55a69ac819ec admin admin] The token cache is being invalidate because project f9febbdff58a49bbb56b0e64aaa851c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:33:36.045910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c86d46b4-7cb4-4955-9f3d-55a69ac819ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:36.046858 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c86d46b4-7cb4-4955-9f3d-55a69ac819ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:36.052004 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1148/2312] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:33:35 2022] DELETE /identity/v3/projects/f9febbdff58a49bbb56b0e64aaa851c7 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:36.067865 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c495e18-37e8-422e-a0d4-78c4b6296c72 service swift] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:36.102617 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c495e18-37e8-422e-a0d4-78c4b6296c72 service swift] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:36.168376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:36.249594 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688dc8f-d734-460b-b2d4-6bcb9fa72641 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:36.250460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688dc8f-d734-460b-b2d4-6bcb9fa72641 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:36.250922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688dc8f-d734-460b-b2d4-6bcb9fa72641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:36.291547 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5688dc8f-d734-460b-b2d4-6bcb9fa72641 None None] MFA Rules not processed for user `aee77a3ed91445ab8b7f58e7f8bc348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:36.293268 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5688dc8f-d734-460b-b2d4-6bcb9fa72641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:36.352457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d6cd1-bf07-462c-b08e-6fd4ae50267f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:36.353419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d6cd1-bf07-462c-b08e-6fd4ae50267f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:36.359046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d6cd1-bf07-462c-b08e-6fd4ae50267f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:36.359352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d6cd1-bf07-462c-b08e-6fd4ae50267f admin admin] PATH_INFO: `/v3/users/8146b6bbd2744671a9f59c0b6c8ce287` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:36.364415 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d6cd1-bf07-462c-b08e-6fd4ae50267f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8146b6bbd2744671a9f59c0b6c8ce287)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:36.374552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d6cd1-bf07-462c-b08e-6fd4ae50267f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:36.474267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-515d6cd1-bf07-462c-b08e-6fd4ae50267f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8146b6bbd2744671a9f59c0b6c8ce287'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:36.475297 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-515d6cd1-bf07-462c-b08e-6fd4ae50267f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8146b6bbd2744671a9f59c0b6c8ce287'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:36.481306 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:36.521401 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-515d6cd1-bf07-462c-b08e-6fd4ae50267f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8146b6bbd2744671a9f59c0b6c8ce287'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:36.523706 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-515d6cd1-bf07-462c-b08e-6fd4ae50267f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8146b6bbd2744671a9f59c0b6c8ce287'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:36.546051 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1149/2313] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 06:33:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:33:36.563111 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688dc8f-d734-460b-b2d4-6bcb9fa72641 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:36.565642 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5688dc8f-d734-460b-b2d4-6bcb9fa72641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:36.567207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-515d6cd1-bf07-462c-b08e-6fd4ae50267f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8146b6bbd2744671a9f59c0b6c8ce287'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:36.601895 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1165/2314] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:33:36 2022] DELETE /identity/v3/users/8146b6bbd2744671a9f59c0b6c8ce287 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:36.784936 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688dc8f-d734-460b-b2d4-6bcb9fa72641 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:37.011152 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21a78f-9c99-4101-9909-0f128786bd27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc205d053d54506bb6c9227e4a1b044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:37.012080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21a78f-9c99-4101-9909-0f128786bd27 service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:37.012383 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21a78f-9c99-4101-9909-0f128786bd27 service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:37.012939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21a78f-9c99-4101-9909-0f128786bd27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:37.183256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21a78f-9c99-4101-9909-0f128786bd27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:37.185049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21a78f-9c99-4101-9909-0f128786bd27 service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:37.221236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1150/2315] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 06:33:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 06:33:37.799530 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f3885-297f-4a2b-af75-f283656e8855 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:37.800884 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f3885-297f-4a2b-af75-f283656e8855 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:37.801495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f3885-297f-4a2b-af75-f283656e8855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:37.834499 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-744f3885-297f-4a2b-af75-f283656e8855 None None] MFA Rules not processed for user `aee77a3ed91445ab8b7f58e7f8bc348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:37.835648 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-744f3885-297f-4a2b-af75-f283656e8855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:37.856180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1166/2316] 10.222.0.65 () {58 vars in 967 bytes} [Thu May 19 06:33:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:37.886097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21a78f-9c99-4101-9909-0f128786bd27 service swift] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:37.900941 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21a78f-9c99-4101-9909-0f128786bd27 service swift] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:37.906994 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256d87d-b935-4914-9ed1-547da4af0f4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc205d053d54506bb6c9227e4a1b044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:37.907886 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256d87d-b935-4914-9ed1-547da4af0f4c service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:37.909049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256d87d-b935-4914-9ed1-547da4af0f4c service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:37.909846 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256d87d-b935-4914-9ed1-547da4af0f4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:37.914590 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f3885-297f-4a2b-af75-f283656e8855 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:37.916405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-744f3885-297f-4a2b-af75-f283656e8855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:37.936964 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256d87d-b935-4914-9ed1-547da4af0f4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:37.941839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256d87d-b935-4914-9ed1-547da4af0f4c service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:37.970065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1151/2317] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 06:33:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 06:33:38.089096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f3885-297f-4a2b-af75-f283656e8855 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:38.121842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2175e6-ce67-49a2-ab56-383d0da345ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:38.132728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2175e6-ce67-49a2-ab56-383d0da345ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:38.132728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2175e6-ce67-49a2-ab56-383d0da345ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:38.132728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2175e6-ce67-49a2-ab56-383d0da345ad admin admin] PATH_INFO: `/v3/projects/a43ddb55909c48c3894a1731d01c7576` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:38.164751 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2175e6-ce67-49a2-ab56-383d0da345ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=a43ddb55909c48c3894a1731d01c7576)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:38.171329 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2175e6-ce67-49a2-ab56-383d0da345ad admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:38.336654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:38.376626 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1d2175e6-ce67-49a2-ab56-383d0da345ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a43ddb55909c48c3894a1731d01c7576', 'request_id': 'req-1d2175e6-ce67-49a2-ab56-383d0da345ad'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:38.401645 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1d2175e6-ce67-49a2-ab56-383d0da345ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a43ddb55909c48c3894a1731d01c7576', 'request_id': 'req-1d2175e6-ce67-49a2-ab56-383d0da345ad'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:38.435555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1d2175e6-ce67-49a2-ab56-383d0da345ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a43ddb55909c48c3894a1731d01c7576', 'request_id': 'req-1d2175e6-ce67-49a2-ab56-383d0da345ad'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:38.466442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1d2175e6-ce67-49a2-ab56-383d0da345ad admin admin] The token cache is being invalidate because project a43ddb55909c48c3894a1731d01c7576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:33:38.466740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1d2175e6-ce67-49a2-ab56-383d0da345ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:38.467438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1d2175e6-ce67-49a2-ab56-383d0da345ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:38.473227 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1167/2318] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:33:37 2022] DELETE /identity/v3/projects/a43ddb55909c48c3894a1731d01c7576 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:33:38.474518 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256d87d-b935-4914-9ed1-547da4af0f4c service swift] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:38.502866 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256d87d-b935-4914-9ed1-547da4af0f4c service swift] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:38.542827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:38.681448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc20e7-021e-4bf5-9b22-1131cabe0c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:38.683362 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc20e7-021e-4bf5-9b22-1131cabe0c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:38.683752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc20e7-021e-4bf5-9b22-1131cabe0c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:38.684138 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc20e7-021e-4bf5-9b22-1131cabe0c49 admin admin] PATH_INFO: `/v3/users/d7fc99e610894930b45deea82fba3df7` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:38.689357 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc20e7-021e-4bf5-9b22-1131cabe0c49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7fc99e610894930b45deea82fba3df7)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:38.691509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc20e7-021e-4bf5-9b22-1131cabe0c49 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:38.785676 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-62fc20e7-021e-4bf5-9b22-1131cabe0c49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7fc99e610894930b45deea82fba3df7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:38.804444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-62fc20e7-021e-4bf5-9b22-1131cabe0c49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7fc99e610894930b45deea82fba3df7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:38.817559 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-62fc20e7-021e-4bf5-9b22-1131cabe0c49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7fc99e610894930b45deea82fba3df7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:38.818414 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-62fc20e7-021e-4bf5-9b22-1131cabe0c49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7fc99e610894930b45deea82fba3df7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:38.821715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-62fc20e7-021e-4bf5-9b22-1131cabe0c49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7fc99e610894930b45deea82fba3df7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:38.841203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1152/2319] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:33:38 2022] DELETE /identity/v3/users/d7fc99e610894930b45deea82fba3df7 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:33:39.792529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2175e6-ce67-49a2-ab56-383d0da345ad admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:39.795457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1d2175e6-ce67-49a2-ab56-383d0da345ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:39.814810 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2175e6-ce67-49a2-ab56-383d0da345ad admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:39.956544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038fcde-f18d-4cd7-8e94-4543280b3a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:39.958752 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038fcde-f18d-4cd7-8e94-4543280b3a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:39.959990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038fcde-f18d-4cd7-8e94-4543280b3a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:39.960444 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038fcde-f18d-4cd7-8e94-4543280b3a6b admin admin] PATH_INFO: `/v3/projects/ccaa6b270e6e492cb7bd2322285fbb1e` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:39.963291 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038fcde-f18d-4cd7-8e94-4543280b3a6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccaa6b270e6e492cb7bd2322285fbb1e)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:39.965264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038fcde-f18d-4cd7-8e94-4543280b3a6b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:39.990256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc20e7-021e-4bf5-9b22-1131cabe0c49 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:39.995712 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-62fc20e7-021e-4bf5-9b22-1131cabe0c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:40.096474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0038fcde-f18d-4cd7-8e94-4543280b3a6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccaa6b270e6e492cb7bd2322285fbb1e', 'request_id': 'req-0038fcde-f18d-4cd7-8e94-4543280b3a6b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:40.104717 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0038fcde-f18d-4cd7-8e94-4543280b3a6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccaa6b270e6e492cb7bd2322285fbb1e', 'request_id': 'req-0038fcde-f18d-4cd7-8e94-4543280b3a6b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:40.117703 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0038fcde-f18d-4cd7-8e94-4543280b3a6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccaa6b270e6e492cb7bd2322285fbb1e', 'request_id': 'req-0038fcde-f18d-4cd7-8e94-4543280b3a6b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:40.147109 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0038fcde-f18d-4cd7-8e94-4543280b3a6b admin admin] The token cache is being invalidate because project ccaa6b270e6e492cb7bd2322285fbb1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:33:40.147452 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0038fcde-f18d-4cd7-8e94-4543280b3a6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:40.149612 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0038fcde-f18d-4cd7-8e94-4543280b3a6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:40.158208 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1168/2320] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:33:39 2022] DELETE /identity/v3/projects/ccaa6b270e6e492cb7bd2322285fbb1e => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:40.161521 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee004db-71bc-44df-b5c7-4a4adb7ad5f3 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:40.162605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee004db-71bc-44df-b5c7-4a4adb7ad5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:40.162910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee004db-71bc-44df-b5c7-4a4adb7ad5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:40.178055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc20e7-021e-4bf5-9b22-1131cabe0c49 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:40.200327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3ee004db-71bc-44df-b5c7-4a4adb7ad5f3 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:40.201919 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3ee004db-71bc-44df-b5c7-4a4adb7ad5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:40.213633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:40.324964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:40.345190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1169/2321] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:40.353364 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32254cf-5cfd-4538-a2cb-f5ebe5508ed0 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:40.353665 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32254cf-5cfd-4538-a2cb-f5ebe5508ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:40.353952 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32254cf-5cfd-4538-a2cb-f5ebe5508ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:40.363166 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13297965-37c4-4689-8c5a-4b571861c63b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b76876d821e4bbbb8d5cad5bd26f559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:40.364001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13297965-37c4-4689-8c5a-4b571861c63b service placement] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:40.365787 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13297965-37c4-4689-8c5a-4b571861c63b service placement] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:40.366205 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13297965-37c4-4689-8c5a-4b571861c63b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:40.374674 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-13297965-37c4-4689-8c5a-4b571861c63b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:40.384266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f32254cf-5cfd-4538-a2cb-f5ebe5508ed0 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:40.387001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f32254cf-5cfd-4538-a2cb-f5ebe5508ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:40.407637 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1170/2322] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:40.409719 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32254cf-5cfd-4538-a2cb-f5ebe5508ed0 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:40.410294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13297965-37c4-4689-8c5a-4b571861c63b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:40.414398 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13297965-37c4-4689-8c5a-4b571861c63b service placement] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:40.426992 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32254cf-5cfd-4538-a2cb-f5ebe5508ed0 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:40.434697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54c51f-388a-40a9-beed-477369bb0b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:40.435621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54c51f-388a-40a9-beed-477369bb0b52 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:40.435913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54c51f-388a-40a9-beed-477369bb0b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:40.436199 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54c51f-388a-40a9-beed-477369bb0b52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:40.436982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54c51f-388a-40a9-beed-477369bb0b52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:33:40.437394 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54c51f-388a-40a9-beed-477369bb0b52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:40.440108 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54c51f-388a-40a9-beed-477369bb0b52 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:40.452150 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1153/2323] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:33:40.461011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1171/2324] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:33:40.462033 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13297965-37c4-4689-8c5a-4b571861c63b service placement] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:40.493008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13297965-37c4-4689-8c5a-4b571861c63b service placement] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:40.505270 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c27ee9-4e38-4492-9a7b-b14bfd7ceed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:40.508530 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c27ee9-4e38-4492-9a7b-b14bfd7ceed8 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:40.510001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c27ee9-4e38-4492-9a7b-b14bfd7ceed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:40.510374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c27ee9-4e38-4492-9a7b-b14bfd7ceed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:40.512590 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c27ee9-4e38-4492-9a7b-b14bfd7ceed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:40.517092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c27ee9-4e38-4492-9a7b-b14bfd7ceed8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:40.590945 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1154/2325] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:33:40 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:33:40.603273 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54c51f-388a-40a9-beed-477369bb0b52 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:40.637547 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54c51f-388a-40a9-beed-477369bb0b52 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:40.798019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eaf914-1c37-4db0-ac60-9b4028ea271f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:40.799369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eaf914-1c37-4db0-ac60-9b4028ea271f admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:40.799369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eaf914-1c37-4db0-ac60-9b4028ea271f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:40.799369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eaf914-1c37-4db0-ac60-9b4028ea271f admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:40.799971 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eaf914-1c37-4db0-ac60-9b4028ea271f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:40.802216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eaf914-1c37-4db0-ac60-9b4028ea271f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:40.847915 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1172/2326] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:33:40 2022] POST /identity/v3/users => generated 292 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:33:40.848559 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c27ee9-4e38-4492-9a7b-b14bfd7ceed8 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:40.877390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c27ee9-4e38-4492-9a7b-b14bfd7ceed8 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:40.886470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c2645-6fe8-4bb6-a1df-7f612194b4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:40.887566 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c2645-6fe8-4bb6-a1df-7f612194b4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:40.888153 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c2645-6fe8-4bb6-a1df-7f612194b4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:40.888887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c2645-6fe8-4bb6-a1df-7f612194b4f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:40.893345 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c2645-6fe8-4bb6-a1df-7f612194b4f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:40.897051 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c2645-6fe8-4bb6-a1df-7f612194b4f4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:40.938536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1155/2327] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:40.940890 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c2645-6fe8-4bb6-a1df-7f612194b4f4 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:40.941781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eaf914-1c37-4db0-ac60-9b4028ea271f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:40.943468 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-311c2645-6fe8-4bb6-a1df-7f612194b4f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:40.961461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eaf914-1c37-4db0-ac60-9b4028ea271f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:40.968325 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c665cbc-b453-4fb3-8f92-b2a2dfa7652d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:40.969470 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c665cbc-b453-4fb3-8f92-b2a2dfa7652d admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:40.970231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c665cbc-b453-4fb3-8f92-b2a2dfa7652d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:40.971686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c2645-6fe8-4bb6-a1df-7f612194b4f4 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:40.971969 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c665cbc-b453-4fb3-8f92-b2a2dfa7652d admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:40.973467 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c665cbc-b453-4fb3-8f92-b2a2dfa7652d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:40.975807 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c665cbc-b453-4fb3-8f92-b2a2dfa7652d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:40.982653 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af84d0-97e0-40c4-b771-948309c7349b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:40.983648 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af84d0-97e0-40c4-b771-948309c7349b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:40.984353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af84d0-97e0-40c4-b771-948309c7349b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:40.984353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af84d0-97e0-40c4-b771-948309c7349b admin admin] PATH_INFO: `/v3/users/b4d720bb1f5c4336a394c413d9d1c612` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:40.995797 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af84d0-97e0-40c4-b771-948309c7349b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4d720bb1f5c4336a394c413d9d1c612)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:40.996451 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af84d0-97e0-40c4-b771-948309c7349b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:40.998290 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1173/2328] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:40.999090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c665cbc-b453-4fb3-8f92-b2a2dfa7652d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:41.023963 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c665cbc-b453-4fb3-8f92-b2a2dfa7652d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:41.043976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52898888-3904-4af4-b830-1a4b7db65b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:41.045439 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52898888-3904-4af4-b830-1a4b7db65b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:41.045742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52898888-3904-4af4-b830-1a4b7db65b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:41.046139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52898888-3904-4af4-b830-1a4b7db65b1f admin admin] PATH_INFO: `/v3/projects/05d7f01134574fdbb0217b60891885d5/users/846adda6dacb46568fd0da03f87ec8aa/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:41.083222 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52898888-3904-4af4-b830-1a4b7db65b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=05d7f01134574fdbb0217b60891885d5, user_id=846adda6dacb46568fd0da03f87ec8aa, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:41.102257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52898888-3904-4af4-b830-1a4b7db65b1f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:41.126809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-36af84d0-97e0-40c4-b771-948309c7349b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4d720bb1f5c4336a394c413d9d1c612'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:41.138651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1174/2329] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:33:40 2022] PUT /identity/v3/projects/05d7f01134574fdbb0217b60891885d5/users/846adda6dacb46568fd0da03f87ec8aa/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:41.151422 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c83a29-da02-45c6-91b9-6c8e7ff3bb98 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:41.152091 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c83a29-da02-45c6-91b9-6c8e7ff3bb98 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:41.152529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c83a29-da02-45c6-91b9-6c8e7ff3bb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:41.157436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-36af84d0-97e0-40c4-b771-948309c7349b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4d720bb1f5c4336a394c413d9d1c612'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:41.183535 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-36af84d0-97e0-40c4-b771-948309c7349b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4d720bb1f5c4336a394c413d9d1c612'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:41.184347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-36af84d0-97e0-40c4-b771-948309c7349b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4d720bb1f5c4336a394c413d9d1c612'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:41.185025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-36af84d0-97e0-40c4-b771-948309c7349b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4d720bb1f5c4336a394c413d9d1c612'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:41.202508 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-95c83a29-da02-45c6-91b9-6c8e7ff3bb98 None None] MFA Rules not processed for user `846adda6dacb46568fd0da03f87ec8aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:41.204155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-95c83a29-da02-45c6-91b9-6c8e7ff3bb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:41.207877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1156/2330] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:33:40 2022] DELETE /identity/v3/users/b4d720bb1f5c4336a394c413d9d1c612 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:33:41.213877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857adce0-fc69-4532-bdaf-e2dff65656a1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:41.214161 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857adce0-fc69-4532-bdaf-e2dff65656a1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:41.214486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857adce0-fc69-4532-bdaf-e2dff65656a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:41.262499 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-857adce0-fc69-4532-bdaf-e2dff65656a1 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:41.263759 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-857adce0-fc69-4532-bdaf-e2dff65656a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:41.331234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1175/2331] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:41.348873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c83a29-da02-45c6-91b9-6c8e7ff3bb98 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:41.350871 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-95c83a29-da02-45c6-91b9-6c8e7ff3bb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:41.361899 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:41.390739 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:41.392323 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1157/2332] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:33:41.486566 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857adce0-fc69-4532-bdaf-e2dff65656a1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:41.489684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-857adce0-fc69-4532-bdaf-e2dff65656a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:41.505385 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c83a29-da02-45c6-91b9-6c8e7ff3bb98 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:41.518025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c11c4-e696-45b0-a9e1-415463fef800 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:41.518775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c11c4-e696-45b0-a9e1-415463fef800 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:41.519673 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c11c4-e696-45b0-a9e1-415463fef800 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:41.524533 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c11c4-e696-45b0-a9e1-415463fef800 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:41.551519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c11c4-e696-45b0-a9e1-415463fef800 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:41.553895 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c11c4-e696-45b0-a9e1-415463fef800 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:41.597116 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1176/2333] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:33:41.669200 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857adce0-fc69-4532-bdaf-e2dff65656a1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:41.678715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868af1e-51f9-47a4-a8d5-4cc7ef6f49d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:41.680477 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868af1e-51f9-47a4-a8d5-4cc7ef6f49d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:41.680810 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868af1e-51f9-47a4-a8d5-4cc7ef6f49d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:41.681101 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868af1e-51f9-47a4-a8d5-4cc7ef6f49d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:41.706137 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6868af1e-51f9-47a4-a8d5-4cc7ef6f49d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:41.711721 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6868af1e-51f9-47a4-a8d5-4cc7ef6f49d9 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:41.740866 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1158/2334] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:33:42.363527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c11c4-e696-45b0-a9e1-415463fef800 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:42.364432 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9c3c11c4-e696-45b0-a9e1-415463fef800 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:42.401255 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c11c4-e696-45b0-a9e1-415463fef800 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:42.415294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63de475-9c13-48b3-88da-630be9a87232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:42.416855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63de475-9c13-48b3-88da-630be9a87232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:42.417151 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63de475-9c13-48b3-88da-630be9a87232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:42.417456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63de475-9c13-48b3-88da-630be9a87232 admin admin] PATH_INFO: `/v3/projects/8beaccf2315245ec8959a58d23ebd293` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:42.420188 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63de475-9c13-48b3-88da-630be9a87232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8beaccf2315245ec8959a58d23ebd293)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:42.428624 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63de475-9c13-48b3-88da-630be9a87232 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:42.592546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e63de475-9c13-48b3-88da-630be9a87232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8beaccf2315245ec8959a58d23ebd293', 'request_id': 'req-e63de475-9c13-48b3-88da-630be9a87232'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:42.604857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e63de475-9c13-48b3-88da-630be9a87232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8beaccf2315245ec8959a58d23ebd293', 'request_id': 'req-e63de475-9c13-48b3-88da-630be9a87232'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:42.623823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e63de475-9c13-48b3-88da-630be9a87232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8beaccf2315245ec8959a58d23ebd293', 'request_id': 'req-e63de475-9c13-48b3-88da-630be9a87232'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:42.676754 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e63de475-9c13-48b3-88da-630be9a87232 admin admin] The token cache is being invalidate because project 8beaccf2315245ec8959a58d23ebd293 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:33:42.676970 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e63de475-9c13-48b3-88da-630be9a87232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:42.678397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-e63de475-9c13-48b3-88da-630be9a87232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:42.681848 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1177/2335] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:33:42 2022] DELETE /identity/v3/projects/8beaccf2315245ec8959a58d23ebd293 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:44.050766 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fa937-6c7c-4756-9001-ad77f585907f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:44.051443 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fa937-6c7c-4756-9001-ad77f585907f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:44.051558 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fa937-6c7c-4756-9001-ad77f585907f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:44.124224 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a65fa937-6c7c-4756-9001-ad77f585907f None None] MFA Rules not processed for user `75ae2c185a764de1864f87a1d109791d`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:44.128671 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a65fa937-6c7c-4756-9001-ad77f585907f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:44.294787 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1159/2336] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:44.307937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63de475-9c13-48b3-88da-630be9a87232 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:44.312459 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e63de475-9c13-48b3-88da-630be9a87232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:44.603763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63de475-9c13-48b3-88da-630be9a87232 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:44.609803 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9b33b-7ba2-4277-b3a9-cc15359b5cdb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a812c3b9b20f44f8814051381a8005b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:44.611397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9b33b-7ba2-4277-b3a9-cc15359b5cdb service glance] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:44.612041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9b33b-7ba2-4277-b3a9-cc15359b5cdb service glance] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:44.612241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9b33b-7ba2-4277-b3a9-cc15359b5cdb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:44.614547 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-59d9b33b-7ba2-4277-b3a9-cc15359b5cdb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:44.712318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:44.746871 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9b33b-7ba2-4277-b3a9-cc15359b5cdb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:44.748567 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9b33b-7ba2-4277-b3a9-cc15359b5cdb service glance] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:44.766838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1178/2337] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:33:44.858692 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fa937-6c7c-4756-9001-ad77f585907f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:44.860141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a65fa937-6c7c-4756-9001-ad77f585907f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:44.962512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c923c-57da-49b5-9163-3fe0ce6a6b54 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:44.962762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c923c-57da-49b5-9163-3fe0ce6a6b54 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:44.963064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c923c-57da-49b5-9163-3fe0ce6a6b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:45.003061 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f15c923c-57da-49b5-9163-3fe0ce6a6b54 None None] MFA Rules not processed for user `aee77a3ed91445ab8b7f58e7f8bc348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:45.005101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f15c923c-57da-49b5-9163-3fe0ce6a6b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:45.008782 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fa937-6c7c-4756-9001-ad77f585907f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:45.014759 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96103f-5576-43ab-a5af-82236f26c055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:45.016526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96103f-5576-43ab-a5af-82236f26c055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:45.018796 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96103f-5576-43ab-a5af-82236f26c055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:45.019297 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96103f-5576-43ab-a5af-82236f26c055 admin admin] PATH_INFO: `/v3/users/846adda6dacb46568fd0da03f87ec8aa` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:45.021349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96103f-5576-43ab-a5af-82236f26c055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=846adda6dacb46568fd0da03f87ec8aa)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:45.026519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96103f-5576-43ab-a5af-82236f26c055 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:45.069541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:45.142473 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8c96103f-5576-43ab-a5af-82236f26c055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '846adda6dacb46568fd0da03f87ec8aa'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:45.175157 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8c96103f-5576-43ab-a5af-82236f26c055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '846adda6dacb46568fd0da03f87ec8aa'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:45.185269 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8c96103f-5576-43ab-a5af-82236f26c055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '846adda6dacb46568fd0da03f87ec8aa'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:45.186696 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8c96103f-5576-43ab-a5af-82236f26c055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '846adda6dacb46568fd0da03f87ec8aa'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:45.187549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8c96103f-5576-43ab-a5af-82236f26c055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '846adda6dacb46568fd0da03f87ec8aa'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:45.206745 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1160/2338] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:33:44 2022] DELETE /identity/v3/users/846adda6dacb46568fd0da03f87ec8aa => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:45.213232 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a968a-1299-4341-b44d-d0c87f8914a1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:45.213598 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a968a-1299-4341-b44d-d0c87f8914a1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:45.213956 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a968a-1299-4341-b44d-d0c87f8914a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:45.227473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1179/2339] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 06:33:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:45.239214 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2f7a968a-1299-4341-b44d-d0c87f8914a1 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:45.240443 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2f7a968a-1299-4341-b44d-d0c87f8914a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:45.243823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c923c-57da-49b5-9163-3fe0ce6a6b54 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:45.245376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f15c923c-57da-49b5-9163-3fe0ce6a6b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:45.338876 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1161/2340] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:45.351022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a968a-1299-4341-b44d-d0c87f8914a1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:45.353524 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2f7a968a-1299-4341-b44d-d0c87f8914a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:45.383447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c923c-57da-49b5-9163-3fe0ce6a6b54 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:45.495312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a968a-1299-4341-b44d-d0c87f8914a1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:45.510141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a4485-80ab-4331-8820-c1af5dc2a558 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:45.511944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a4485-80ab-4331-8820-c1af5dc2a558 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:45.512237 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a4485-80ab-4331-8820-c1af5dc2a558 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:45.512516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a4485-80ab-4331-8820-c1af5dc2a558 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:45.519985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082d307-cb02-49b7-9c7a-d1a89574c0b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc205d053d54506bb6c9227e4a1b044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:45.520554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082d307-cb02-49b7-9c7a-d1a89574c0b2 service swift] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:45.520839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082d307-cb02-49b7-9c7a-d1a89574c0b2 service swift] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:45.521122 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082d307-cb02-49b7-9c7a-d1a89574c0b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:45.530260 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a4485-80ab-4331-8820-c1af5dc2a558 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:45.532394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a4485-80ab-4331-8820-c1af5dc2a558 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:45.550534 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1162/2341] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:33:45.570931 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:45.620236 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0082d307-cb02-49b7-9c7a-d1a89574c0b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:45.624137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0082d307-cb02-49b7-9c7a-d1a89574c0b2 service swift] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:45.635833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1180/2342] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 06:33:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 06:33:46.048558 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a4485-80ab-4331-8820-c1af5dc2a558 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:46.050682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c21a4485-80ab-4331-8820-c1af5dc2a558 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:46.080055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a4485-80ab-4331-8820-c1af5dc2a558 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:46.088431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb728b-e124-412d-b923-c2e3f00ffd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:46.090092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb728b-e124-412d-b923-c2e3f00ffd81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:46.090495 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb728b-e124-412d-b923-c2e3f00ffd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:46.090864 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb728b-e124-412d-b923-c2e3f00ffd81 admin admin] PATH_INFO: `/v3/projects/05d7f01134574fdbb0217b60891885d5` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:46.092268 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb728b-e124-412d-b923-c2e3f00ffd81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05d7f01134574fdbb0217b60891885d5)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:46.094485 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb728b-e124-412d-b923-c2e3f00ffd81 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:46.156230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:46.171657 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d0eb728b-e124-412d-b923-c2e3f00ffd81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05d7f01134574fdbb0217b60891885d5', 'request_id': 'req-d0eb728b-e124-412d-b923-c2e3f00ffd81'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:46.184750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d0eb728b-e124-412d-b923-c2e3f00ffd81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05d7f01134574fdbb0217b60891885d5', 'request_id': 'req-d0eb728b-e124-412d-b923-c2e3f00ffd81'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:46.222630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d0eb728b-e124-412d-b923-c2e3f00ffd81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05d7f01134574fdbb0217b60891885d5', 'request_id': 'req-d0eb728b-e124-412d-b923-c2e3f00ffd81'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:46.239016 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d0eb728b-e124-412d-b923-c2e3f00ffd81 admin admin] The token cache is being invalidate because project 05d7f01134574fdbb0217b60891885d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:33:46.239693 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d0eb728b-e124-412d-b923-c2e3f00ffd81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:46.240624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d0eb728b-e124-412d-b923-c2e3f00ffd81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:33:46.243150 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1163/2343] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:33:46 2022] DELETE /identity/v3/projects/05d7f01134574fdbb0217b60891885d5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:46.247999 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3f255-6b0a-4898-941d-9c9c321ee577 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:46.248289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3f255-6b0a-4898-941d-9c9c321ee577 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:46.248597 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3f255-6b0a-4898-941d-9c9c321ee577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:46.271994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-57e3f255-6b0a-4898-941d-9c9c321ee577 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:46.273241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-57e3f255-6b0a-4898-941d-9c9c321ee577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:46.388526 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1181/2344] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:46.402596 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd3f0f-ed36-48fa-85f2-c8952e6babbd None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:46.403017 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd3f0f-ed36-48fa-85f2-c8952e6babbd None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:46.403690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd3f0f-ed36-48fa-85f2-c8952e6babbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:46.435575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4dfd3f0f-ed36-48fa-85f2-c8952e6babbd None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:46.437409 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4dfd3f0f-ed36-48fa-85f2-c8952e6babbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:46.452644 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1164/2345] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:46.453793 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3f255-6b0a-4898-941d-9c9c321ee577 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:46.470254 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3f255-6b0a-4898-941d-9c9c321ee577 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:46.478938 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50956ac6-9aa8-43da-a77d-8ef57b953b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:46.479681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50956ac6-9aa8-43da-a77d-8ef57b953b1a admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:46.479970 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50956ac6-9aa8-43da-a77d-8ef57b953b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:46.480253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50956ac6-9aa8-43da-a77d-8ef57b953b1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:46.481001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50956ac6-9aa8-43da-a77d-8ef57b953b1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:33:46.481585 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50956ac6-9aa8-43da-a77d-8ef57b953b1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:46.493529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50956ac6-9aa8-43da-a77d-8ef57b953b1a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:46.514136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1182/2346] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:33:46.518784 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd3f0f-ed36-48fa-85f2-c8952e6babbd None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:46.530038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd3f0f-ed36-48fa-85f2-c8952e6babbd None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:46.537675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d93573-d286-418a-a492-b8d142182749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:46.540182 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d93573-d286-418a-a492-b8d142182749 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:46.540548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d93573-d286-418a-a492-b8d142182749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:46.540898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d93573-d286-418a-a492-b8d142182749 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:46.545096 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d93573-d286-418a-a492-b8d142182749 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:46.548337 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d93573-d286-418a-a492-b8d142182749 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:46.606307 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1165/2347] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:33:46 2022] POST /identity/v3/projects => generated 363 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:33:46.610622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50956ac6-9aa8-43da-a77d-8ef57b953b1a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:46.625068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50956ac6-9aa8-43da-a77d-8ef57b953b1a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:46.743985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1aac8-1cca-462e-b1a9-12d63e6cecd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:46.744592 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1aac8-1cca-462e-b1a9-12d63e6cecd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:46.744875 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1aac8-1cca-462e-b1a9-12d63e6cecd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:46.745152 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1aac8-1cca-462e-b1a9-12d63e6cecd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:46.745891 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1aac8-1cca-462e-b1a9-12d63e6cecd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:46.748385 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1aac8-1cca-462e-b1a9-12d63e6cecd6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:46.785056 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1183/2348] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:33:46 2022] POST /identity/v3/users => generated 282 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:33:46.788621 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d93573-d286-418a-a492-b8d142182749 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:46.810228 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d93573-d286-418a-a492-b8d142182749 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:46.820484 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30e339-eeb3-46dc-8f4d-eaaaaac6f39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:46.821364 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30e339-eeb3-46dc-8f4d-eaaaaac6f39b admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:46.821689 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30e339-eeb3-46dc-8f4d-eaaaaac6f39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:46.823414 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30e339-eeb3-46dc-8f4d-eaaaaac6f39b admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:46.825072 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30e339-eeb3-46dc-8f4d-eaaaaac6f39b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:46.829171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30e339-eeb3-46dc-8f4d-eaaaaac6f39b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:46.843336 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1166/2349] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:46.848078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1aac8-1cca-462e-b1a9-12d63e6cecd6 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:46.863501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1aac8-1cca-462e-b1a9-12d63e6cecd6 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:46.898458 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06953381-eea0-4185-befa-3402964e49d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:46.899384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06953381-eea0-4185-befa-3402964e49d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:46.900395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06953381-eea0-4185-befa-3402964e49d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:46.901360 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06953381-eea0-4185-befa-3402964e49d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:46.902448 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06953381-eea0-4185-befa-3402964e49d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:46.904747 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06953381-eea0-4185-befa-3402964e49d2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:46.927090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1184/2350] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:33:46.931242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30e339-eeb3-46dc-8f4d-eaaaaac6f39b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:46.943200 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30e339-eeb3-46dc-8f4d-eaaaaac6f39b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:46.953321 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76e7d9-046d-4887-a706-1105942ba2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:46.954248 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76e7d9-046d-4887-a706-1105942ba2dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:46.954588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76e7d9-046d-4887-a706-1105942ba2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:46.954839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76e7d9-046d-4887-a706-1105942ba2dc admin admin] PATH_INFO: `/v3/projects/2b9d07369a2f47e486be7cf5cd68f9a5/users/3a698e85a3b24bc29b11c58e6f82e1e1/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:47.091840 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76e7d9-046d-4887-a706-1105942ba2dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b9d07369a2f47e486be7cf5cd68f9a5, user_id=3a698e85a3b24bc29b11c58e6f82e1e1, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:47.096118 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76e7d9-046d-4887-a706-1105942ba2dc admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:47.140487 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1167/2351] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:33:46 2022] PUT /identity/v3/projects/2b9d07369a2f47e486be7cf5cd68f9a5/users/3a698e85a3b24bc29b11c58e6f82e1e1/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:33:47.150860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83c71e-1e67-4835-b909-6b23eb878e25 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:47.151155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83c71e-1e67-4835-b909-6b23eb878e25 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:47.151437 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83c71e-1e67-4835-b909-6b23eb878e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:47.223269 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-fd83c71e-1e67-4835-b909-6b23eb878e25 None None] MFA Rules not processed for user `3a698e85a3b24bc29b11c58e6f82e1e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:47.225267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fd83c71e-1e67-4835-b909-6b23eb878e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:47.295408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:47.411082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1185/2352] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:47 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:33:47.429970 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76e7d9-046d-4887-a706-1105942ba2dc admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:47.432547 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0b76e7d9-046d-4887-a706-1105942ba2dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:47.557060 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76e7d9-046d-4887-a706-1105942ba2dc admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:47.567251 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51f530-515e-4541-8faa-e2d459c11b45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:47.568260 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51f530-515e-4541-8faa-e2d459c11b45 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:47.568656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51f530-515e-4541-8faa-e2d459c11b45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:47.569037 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51f530-515e-4541-8faa-e2d459c11b45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:47.601994 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af51f530-515e-4541-8faa-e2d459c11b45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:47.623540 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af51f530-515e-4541-8faa-e2d459c11b45 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:47.651384 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1168/2353] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:33:47 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:33:48.939405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dc32e-8806-4e6a-b38b-6e385285a1ab None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:48.959581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dc32e-8806-4e6a-b38b-6e385285a1ab None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:48.960426 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dc32e-8806-4e6a-b38b-6e385285a1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:49.005765 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-108dc32e-8806-4e6a-b38b-6e385285a1ab None None] MFA Rules not processed for user `aee77a3ed91445ab8b7f58e7f8bc348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:49.008521 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-108dc32e-8806-4e6a-b38b-6e385285a1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:49.172075 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1186/2354] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 06:33:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:49.187392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51f530-515e-4541-8faa-e2d459c11b45 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:33:49.192265 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-af51f530-515e-4541-8faa-e2d459c11b45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:49.366024 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:33:49.374373 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51f530-515e-4541-8faa-e2d459c11b45 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:33:49.389634 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a245165-2b96-455f-a812-100593308b82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc205d053d54506bb6c9227e4a1b044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:33:49.392570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a245165-2b96-455f-a812-100593308b82 service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:49.393414 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a245165-2b96-455f-a812-100593308b82 service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:49.394091 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a245165-2b96-455f-a812-100593308b82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:49.417562 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a245165-2b96-455f-a812-100593308b82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:33:49.417562 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a245165-2b96-455f-a812-100593308b82 service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:33:49.437526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1169/2355] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 06:33:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 06:33:52.330987 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699a61f-fe3b-43a8-afb8-1450ee8d89b0 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:52.332277 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699a61f-fe3b-43a8-afb8-1450ee8d89b0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:52.332740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699a61f-fe3b-43a8-afb8-1450ee8d89b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:52.368847 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8699a61f-fe3b-43a8-afb8-1450ee8d89b0 None None] MFA Rules not processed for user `6a71c3c016564c3b9ab0b6b58d378fda`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:52.370632 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8699a61f-fe3b-43a8-afb8-1450ee8d89b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:52.510058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1187/2356] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:33:59.334809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236d2ca-f113-42ff-8851-6d0b20650e83 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:33:59.334809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236d2ca-f113-42ff-8851-6d0b20650e83 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:33:59.334809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236d2ca-f113-42ff-8851-6d0b20650e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:33:59.356657 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-8236d2ca-f113-42ff-8851-6d0b20650e83 None None] MFA Rules not processed for user `d4008c6bb57b4a91b2761fc87a1e9d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:33:59.360959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8236d2ca-f113-42ff-8851-6d0b20650e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:33:59.457335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1170/2357] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:33:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:34:02.494866 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=66 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:34:02.911799 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552aa71-6956-48e8-a63c-cfd721c6217f None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:02.912100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552aa71-6956-48e8-a63c-cfd721c6217f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:02.912391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552aa71-6956-48e8-a63c-cfd721c6217f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:02.935762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-9552aa71-6956-48e8-a63c-cfd721c6217f None None] MFA Rules not processed for user `aee77a3ed91445ab8b7f58e7f8bc348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:34:02.937233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9552aa71-6956-48e8-a63c-cfd721c6217f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:02.945437 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1188/2358] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 06:34:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:34:02.980048 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236d2ca-f113-42ff-8851-6d0b20650e83 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:03.000211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236d2ca-f113-42ff-8851-6d0b20650e83 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:03.011682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b15c6a-ae08-4ae5-9433-b866110f9344 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc205d053d54506bb6c9227e4a1b044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:03.012736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b15c6a-ae08-4ae5-9433-b866110f9344 service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:03.013184 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b15c6a-ae08-4ae5-9433-b866110f9344 service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:03.013645 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b15c6a-ae08-4ae5-9433-b866110f9344 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:03.039286 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b15c6a-ae08-4ae5-9433-b866110f9344 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:03.041347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b15c6a-ae08-4ae5-9433-b866110f9344 service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:03.060313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1171/2359] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 06:34:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 06:34:12.925651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:34:13.056412 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=35 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:34:15.018079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e997e10-d2c3-45fd-8f41-c74f10053cb4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:15.018660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e997e10-d2c3-45fd-8f41-c74f10053cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:15.018795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e997e10-d2c3-45fd-8f41-c74f10053cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:15.044158 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3e997e10-d2c3-45fd-8f41-c74f10053cb4 None None] MFA Rules not processed for user `aee77a3ed91445ab8b7f58e7f8bc348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:34:15.045416 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3e997e10-d2c3-45fd-8f41-c74f10053cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:15.054935 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1189/2360] 10.222.0.65 () {58 vars in 967 bytes} [Thu May 19 06:34:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:34:15.069957 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b15c6a-ae08-4ae5-9433-b866110f9344 service swift] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:15.081204 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b15c6a-ae08-4ae5-9433-b866110f9344 service swift] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:15.086538 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f1fac-bc88-47a7-9597-539e315a29fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc205d053d54506bb6c9227e4a1b044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:15.087199 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f1fac-bc88-47a7-9597-539e315a29fc service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:15.087491 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f1fac-bc88-47a7-9597-539e315a29fc service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:15.087776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f1fac-bc88-47a7-9597-539e315a29fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:15.097594 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f1fac-bc88-47a7-9597-539e315a29fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:15.099144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f1fac-bc88-47a7-9597-539e315a29fc service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:15.107659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1172/2361] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 06:34:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 06:34:17.572012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e997e10-d2c3-45fd-8f41-c74f10053cb4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:17.573512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3e997e10-d2c3-45fd-8f41-c74f10053cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:17.671270 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e997e10-d2c3-45fd-8f41-c74f10053cb4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:17.676973 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094ee2d-0dc5-4343-9ad4-426e457d2319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:17.677915 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094ee2d-0dc5-4343-9ad4-426e457d2319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:17.678206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094ee2d-0dc5-4343-9ad4-426e457d2319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:17.678487 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094ee2d-0dc5-4343-9ad4-426e457d2319 admin admin] PATH_INFO: `/v3/users/ce2c7ead49a047c6bc04a783ed0bfc43` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:17.679788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094ee2d-0dc5-4343-9ad4-426e457d2319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce2c7ead49a047c6bc04a783ed0bfc43)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:17.681829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094ee2d-0dc5-4343-9ad4-426e457d2319 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:17.737523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d094ee2d-0dc5-4343-9ad4-426e457d2319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce2c7ead49a047c6bc04a783ed0bfc43'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:17.738194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d094ee2d-0dc5-4343-9ad4-426e457d2319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce2c7ead49a047c6bc04a783ed0bfc43'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:17.751178 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d094ee2d-0dc5-4343-9ad4-426e457d2319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce2c7ead49a047c6bc04a783ed0bfc43'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:17.751859 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d094ee2d-0dc5-4343-9ad4-426e457d2319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce2c7ead49a047c6bc04a783ed0bfc43'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:17.759406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d094ee2d-0dc5-4343-9ad4-426e457d2319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce2c7ead49a047c6bc04a783ed0bfc43'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:17.774965 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1190/2362] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:34:17 2022] DELETE /identity/v3/users/ce2c7ead49a047c6bc04a783ed0bfc43 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:34:17.967992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f1fac-bc88-47a7-9597-539e315a29fc service swift] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:17.979944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f1fac-bc88-47a7-9597-539e315a29fc service swift] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:18.091907 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a150a-1dc0-4873-9f64-b8ae6cbc1af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:18.093815 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a150a-1dc0-4873-9f64-b8ae6cbc1af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:18.094663 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a150a-1dc0-4873-9f64-b8ae6cbc1af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:18.095146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a150a-1dc0-4873-9f64-b8ae6cbc1af2 admin admin] PATH_INFO: `/v3/projects/af231f2d98544631857757d5b48fd918` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:18.097015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a150a-1dc0-4873-9f64-b8ae6cbc1af2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af231f2d98544631857757d5b48fd918)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:18.099264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a150a-1dc0-4873-9f64-b8ae6cbc1af2 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:18.149594 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-941a150a-1dc0-4873-9f64-b8ae6cbc1af2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af231f2d98544631857757d5b48fd918', 'request_id': 'req-941a150a-1dc0-4873-9f64-b8ae6cbc1af2'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:18.157675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-941a150a-1dc0-4873-9f64-b8ae6cbc1af2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af231f2d98544631857757d5b48fd918', 'request_id': 'req-941a150a-1dc0-4873-9f64-b8ae6cbc1af2'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:18.163265 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-941a150a-1dc0-4873-9f64-b8ae6cbc1af2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af231f2d98544631857757d5b48fd918', 'request_id': 'req-941a150a-1dc0-4873-9f64-b8ae6cbc1af2'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:18.174884 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-941a150a-1dc0-4873-9f64-b8ae6cbc1af2 admin admin] The token cache is being invalidate because project af231f2d98544631857757d5b48fd918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:34:18.175166 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-941a150a-1dc0-4873-9f64-b8ae6cbc1af2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:18.175904 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-941a150a-1dc0-4873-9f64-b8ae6cbc1af2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:18.177612 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1173/2363] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:34:17 2022] DELETE /identity/v3/projects/af231f2d98544631857757d5b48fd918 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:34:18.184387 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a280ea6-dce4-42d8-b5ae-9727e902a989 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:18.184685 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a280ea6-dce4-42d8-b5ae-9727e902a989 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:18.184969 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a280ea6-dce4-42d8-b5ae-9727e902a989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:18.204442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-7a280ea6-dce4-42d8-b5ae-9727e902a989 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:34:18.205702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7a280ea6-dce4-42d8-b5ae-9727e902a989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:18.278785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1191/2364] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:34:18.287180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203a032-a1e8-4ffa-bf43-3f956b5ce2b7 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:18.287568 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203a032-a1e8-4ffa-bf43-3f956b5ce2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:18.288065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203a032-a1e8-4ffa-bf43-3f956b5ce2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:18.307155 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-8203a032-a1e8-4ffa-bf43-3f956b5ce2b7 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:34:18.308110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8203a032-a1e8-4ffa-bf43-3f956b5ce2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:18.316832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1174/2365] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:34:18.323250 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a280ea6-dce4-42d8-b5ae-9727e902a989 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:18.343260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a280ea6-dce4-42d8-b5ae-9727e902a989 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:18.349329 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b269166-4523-4d5a-a9c1-fb87ac0a8373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:18.350011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b269166-4523-4d5a-a9c1-fb87ac0a8373 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:18.350297 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b269166-4523-4d5a-a9c1-fb87ac0a8373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:18.350627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b269166-4523-4d5a-a9c1-fb87ac0a8373 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:18.351354 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b269166-4523-4d5a-a9c1-fb87ac0a8373 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:34:18.351657 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b269166-4523-4d5a-a9c1-fb87ac0a8373 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:18.353427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b269166-4523-4d5a-a9c1-fb87ac0a8373 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:18.362671 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:34:18.366686 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1192/2366] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:34:18.370919 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203a032-a1e8-4ffa-bf43-3f956b5ce2b7 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:18.380446 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203a032-a1e8-4ffa-bf43-3f956b5ce2b7 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:18.386036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16585dca-c567-4547-8634-70215d912fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:18.386691 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16585dca-c567-4547-8634-70215d912fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:18.386981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16585dca-c567-4547-8634-70215d912fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:18.387264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16585dca-c567-4547-8634-70215d912fb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:18.387957 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16585dca-c567-4547-8634-70215d912fb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:18.389923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16585dca-c567-4547-8634-70215d912fb6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:18.419650 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1175/2367] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:34:18 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:34:18.420820 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b269166-4523-4d5a-a9c1-fb87ac0a8373 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:18.431445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b269166-4523-4d5a-a9c1-fb87ac0a8373 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:18.490976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa51c2-2c6c-4d76-ac72-d8ccdf0080fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:18.491666 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa51c2-2c6c-4d76-ac72-d8ccdf0080fc admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:18.491950 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa51c2-2c6c-4d76-ac72-d8ccdf0080fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:18.492239 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa51c2-2c6c-4d76-ac72-d8ccdf0080fc admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:18.492949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa51c2-2c6c-4d76-ac72-d8ccdf0080fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:18.494907 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa51c2-2c6c-4d76-ac72-d8ccdf0080fc admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:18.517618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1193/2368] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:34:18 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:34:18.522452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16585dca-c567-4547-8634-70215d912fb6 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:18.536189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16585dca-c567-4547-8634-70215d912fb6 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:18.541134 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268a11a-c1a8-4c1a-b3f7-07ddea62d84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:18.541859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268a11a-c1a8-4c1a-b3f7-07ddea62d84d admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:18.542233 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268a11a-c1a8-4c1a-b3f7-07ddea62d84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:18.542526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268a11a-c1a8-4c1a-b3f7-07ddea62d84d admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:18.543285 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268a11a-c1a8-4c1a-b3f7-07ddea62d84d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:18.544935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268a11a-c1a8-4c1a-b3f7-07ddea62d84d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:18.556984 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1176/2369] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:34:18.562324 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa51c2-2c6c-4d76-ac72-d8ccdf0080fc admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:18.580413 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa51c2-2c6c-4d76-ac72-d8ccdf0080fc admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:18.587211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ea40a-1dba-45e7-88d1-9736397f8894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:18.588136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ea40a-1dba-45e7-88d1-9736397f8894 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:18.588547 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ea40a-1dba-45e7-88d1-9736397f8894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:18.588893 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ea40a-1dba-45e7-88d1-9736397f8894 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:18.589781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ea40a-1dba-45e7-88d1-9736397f8894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:18.591857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ea40a-1dba-45e7-88d1-9736397f8894 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:18.608857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1194/2370] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:34:18.610640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268a11a-c1a8-4c1a-b3f7-07ddea62d84d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:18.626334 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268a11a-c1a8-4c1a-b3f7-07ddea62d84d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:18.634110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f817127a-e6e6-4084-a618-be473b31c0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:18.635967 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817127a-e6e6-4084-a618-be473b31c0c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:18.636439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817127a-e6e6-4084-a618-be473b31c0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:18.636892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817127a-e6e6-4084-a618-be473b31c0c5 admin admin] PATH_INFO: `/v3/projects/af1d5b8b00b74c88a836613465585c42/users/b358dc1ebb56475290dd002f48b45f81/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:18.657989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f817127a-e6e6-4084-a618-be473b31c0c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af1d5b8b00b74c88a836613465585c42, user_id=b358dc1ebb56475290dd002f48b45f81, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:18.660703 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f817127a-e6e6-4084-a618-be473b31c0c5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:18.676192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1177/2371] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:34:18 2022] PUT /identity/v3/projects/af1d5b8b00b74c88a836613465585c42/users/b358dc1ebb56475290dd002f48b45f81/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:34:18.686781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b51ae5-20d9-4ae7-875e-c688f0269ce2 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:18.687072 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b51ae5-20d9-4ae7-875e-c688f0269ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:18.687534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b51ae5-20d9-4ae7-875e-c688f0269ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:18.716935 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a0b51ae5-20d9-4ae7-875e-c688f0269ce2 None None] MFA Rules not processed for user `b358dc1ebb56475290dd002f48b45f81`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:34:18.718225 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a0b51ae5-20d9-4ae7-875e-c688f0269ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:18.819115 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1195/2372] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:34:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:34:18.834706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f817127a-e6e6-4084-a618-be473b31c0c5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:18.836533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f817127a-e6e6-4084-a618-be473b31c0c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:18.855314 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:34:18.923910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f817127a-e6e6-4084-a618-be473b31c0c5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:18.929276 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e82f9-e4dd-446d-aca8-2b74714be24c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a812c3b9b20f44f8814051381a8005b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:18.929976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e82f9-e4dd-446d-aca8-2b74714be24c service glance] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:18.929976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e82f9-e4dd-446d-aca8-2b74714be24c service glance] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:18.930625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e82f9-e4dd-446d-aca8-2b74714be24c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:18.940869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e82f9-e4dd-446d-aca8-2b74714be24c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:18.942719 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e82f9-e4dd-446d-aca8-2b74714be24c service glance] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:18.959615 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1178/2373] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:34:18 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:34:28.815026 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:34:28.951620 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=20 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:34:34.530988 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b51ae5-20d9-4ae7-875e-c688f0269ce2 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:34.532486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a0b51ae5-20d9-4ae7-875e-c688f0269ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:34.601516 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b51ae5-20d9-4ae7-875e-c688f0269ce2 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:34.606277 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529beb8c-31aa-447e-b0e7-7528e13c7cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:34.607164 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529beb8c-31aa-447e-b0e7-7528e13c7cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:34.607446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529beb8c-31aa-447e-b0e7-7528e13c7cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:34.607725 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529beb8c-31aa-447e-b0e7-7528e13c7cd4 admin admin] PATH_INFO: `/v3/users/3a698e85a3b24bc29b11c58e6f82e1e1` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:34.609460 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529beb8c-31aa-447e-b0e7-7528e13c7cd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a698e85a3b24bc29b11c58e6f82e1e1)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:34.611419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529beb8c-31aa-447e-b0e7-7528e13c7cd4 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:34.665006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-529beb8c-31aa-447e-b0e7-7528e13c7cd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a698e85a3b24bc29b11c58e6f82e1e1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:34.665763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-529beb8c-31aa-447e-b0e7-7528e13c7cd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a698e85a3b24bc29b11c58e6f82e1e1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:34.683351 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-529beb8c-31aa-447e-b0e7-7528e13c7cd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a698e85a3b24bc29b11c58e6f82e1e1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:34.684075 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-529beb8c-31aa-447e-b0e7-7528e13c7cd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a698e85a3b24bc29b11c58e6f82e1e1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:34.692838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-529beb8c-31aa-447e-b0e7-7528e13c7cd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a698e85a3b24bc29b11c58e6f82e1e1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:34.705961 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1196/2374] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:34:34 2022] DELETE /identity/v3/users/3a698e85a3b24bc29b11c58e6f82e1e1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:34:34.710682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b52804-0e99-48a4-97be-3f5253219694 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:34.711284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b52804-0e99-48a4-97be-3f5253219694 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:34.711284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b52804-0e99-48a4-97be-3f5253219694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:34.738741 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a0b52804-0e99-48a4-97be-3f5253219694 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:34:34.740007 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a0b52804-0e99-48a4-97be-3f5253219694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:34.818750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1179/2375] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:34:34.828346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529beb8c-31aa-447e-b0e7-7528e13c7cd4 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:34.829810 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-529beb8c-31aa-447e-b0e7-7528e13c7cd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:34.888766 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529beb8c-31aa-447e-b0e7-7528e13c7cd4 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:34.894066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3d72e-98d0-4eea-b56f-d85703143bfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:34.894733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3d72e-98d0-4eea-b56f-d85703143bfe service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:34.895020 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3d72e-98d0-4eea-b56f-d85703143bfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:34.895301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3d72e-98d0-4eea-b56f-d85703143bfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:34.905441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3d72e-98d0-4eea-b56f-d85703143bfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:34.907449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3d72e-98d0-4eea-b56f-d85703143bfe service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:34.917730 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1197/2376] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:34:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:34:35.246266 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b52804-0e99-48a4-97be-3f5253219694 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:35.247862 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a0b52804-0e99-48a4-97be-3f5253219694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:35.266708 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b52804-0e99-48a4-97be-3f5253219694 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:35.272003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd883b50-125c-4d8e-b4c9-a6f3025430fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:35.273074 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd883b50-125c-4d8e-b4c9-a6f3025430fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:35.273392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd883b50-125c-4d8e-b4c9-a6f3025430fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:35.273578 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd883b50-125c-4d8e-b4c9-a6f3025430fa admin admin] PATH_INFO: `/v3/projects/2b9d07369a2f47e486be7cf5cd68f9a5` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:35.275004 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd883b50-125c-4d8e-b4c9-a6f3025430fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b9d07369a2f47e486be7cf5cd68f9a5)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:35.277119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd883b50-125c-4d8e-b4c9-a6f3025430fa admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:35.335141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bd883b50-125c-4d8e-b4c9-a6f3025430fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b9d07369a2f47e486be7cf5cd68f9a5', 'request_id': 'req-bd883b50-125c-4d8e-b4c9-a6f3025430fa'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:35.345662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bd883b50-125c-4d8e-b4c9-a6f3025430fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b9d07369a2f47e486be7cf5cd68f9a5', 'request_id': 'req-bd883b50-125c-4d8e-b4c9-a6f3025430fa'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:35.351396 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bd883b50-125c-4d8e-b4c9-a6f3025430fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b9d07369a2f47e486be7cf5cd68f9a5', 'request_id': 'req-bd883b50-125c-4d8e-b4c9-a6f3025430fa'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:35.364190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bd883b50-125c-4d8e-b4c9-a6f3025430fa admin admin] The token cache is being invalidate because project 2b9d07369a2f47e486be7cf5cd68f9a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:34:35.364482 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bd883b50-125c-4d8e-b4c9-a6f3025430fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:35.365132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bd883b50-125c-4d8e-b4c9-a6f3025430fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:35.366713 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1180/2377] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:34:35 2022] DELETE /identity/v3/projects/2b9d07369a2f47e486be7cf5cd68f9a5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:34:35.371656 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b530209-37b2-4363-8ad7-a7231ac0fe5a None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:35.371950 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b530209-37b2-4363-8ad7-a7231ac0fe5a None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:35.372216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b530209-37b2-4363-8ad7-a7231ac0fe5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:35.390601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-7b530209-37b2-4363-8ad7-a7231ac0fe5a None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:34:35.391800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7b530209-37b2-4363-8ad7-a7231ac0fe5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:35.409658 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:34:35.457326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1198/2378] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:34:35.467528 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb65d6-9606-4301-b156-459a6a4b7b6f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:35.467836 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb65d6-9606-4301-b156-459a6a4b7b6f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:35.469186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb65d6-9606-4301-b156-459a6a4b7b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:35.488360 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-fecb65d6-9606-4301-b156-459a6a4b7b6f None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:34:35.489634 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fecb65d6-9606-4301-b156-459a6a4b7b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:35.498333 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1181/2379] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:34:35.503156 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b530209-37b2-4363-8ad7-a7231ac0fe5a None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:35.512942 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b530209-37b2-4363-8ad7-a7231ac0fe5a None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:35.518828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459e7c6-7324-4fa3-b45a-652a83c96ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:35.519523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459e7c6-7324-4fa3-b45a-652a83c96ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:35.519812 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459e7c6-7324-4fa3-b45a-652a83c96ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:35.520091 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459e7c6-7324-4fa3-b45a-652a83c96ca3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:35.520827 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459e7c6-7324-4fa3-b45a-652a83c96ca3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:34:35.521132 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459e7c6-7324-4fa3-b45a-652a83c96ca3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:35.522767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459e7c6-7324-4fa3-b45a-652a83c96ca3 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:35.535268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1199/2380] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:34:35.539628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb65d6-9606-4301-b156-459a6a4b7b6f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:35.550141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb65d6-9606-4301-b156-459a6a4b7b6f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:35.555847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18699d-e356-4f74-b258-5c03e54a7527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:35.556613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18699d-e356-4f74-b258-5c03e54a7527 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:35.556910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18699d-e356-4f74-b258-5c03e54a7527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:35.557192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18699d-e356-4f74-b258-5c03e54a7527 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:35.558040 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18699d-e356-4f74-b258-5c03e54a7527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:35.559960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18699d-e356-4f74-b258-5c03e54a7527 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:35.588055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1182/2381] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:34:35 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:34:35.594780 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459e7c6-7324-4fa3-b45a-652a83c96ca3 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:35.609371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459e7c6-7324-4fa3-b45a-652a83c96ca3 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:35.671764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5eec52-1896-4ec7-a9eb-cd454ba8f2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:35.672455 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eec52-1896-4ec7-a9eb-cd454ba8f2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:35.672738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eec52-1896-4ec7-a9eb-cd454ba8f2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:35.673019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eec52-1896-4ec7-a9eb-cd454ba8f2e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:35.673753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5eec52-1896-4ec7-a9eb-cd454ba8f2e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:35.675678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5eec52-1896-4ec7-a9eb-cd454ba8f2e5 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:35.702426 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1200/2382] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:34:35 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:34:35.706971 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18699d-e356-4f74-b258-5c03e54a7527 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:35.717875 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18699d-e356-4f74-b258-5c03e54a7527 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:35.723019 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ce29f-cf7c-46b9-9420-1593945fa94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:35.723715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ce29f-cf7c-46b9-9420-1593945fa94d admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:35.724001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ce29f-cf7c-46b9-9420-1593945fa94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:35.724285 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ce29f-cf7c-46b9-9420-1593945fa94d admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:35.725015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ce29f-cf7c-46b9-9420-1593945fa94d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:35.726678 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ce29f-cf7c-46b9-9420-1593945fa94d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:35.740588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1183/2383] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:34:35.741756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5eec52-1896-4ec7-a9eb-cd454ba8f2e5 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:35.752822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5eec52-1896-4ec7-a9eb-cd454ba8f2e5 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:35.757600 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620abbb4-7d1e-44ef-bd3a-f4b199b0598d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:35.758278 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620abbb4-7d1e-44ef-bd3a-f4b199b0598d admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:35.758564 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620abbb4-7d1e-44ef-bd3a-f4b199b0598d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:35.758845 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620abbb4-7d1e-44ef-bd3a-f4b199b0598d admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:35.759585 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620abbb4-7d1e-44ef-bd3a-f4b199b0598d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:35.761191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620abbb4-7d1e-44ef-bd3a-f4b199b0598d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:35.772174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1201/2384] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:34:35.776393 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ce29f-cf7c-46b9-9420-1593945fa94d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:35.786697 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ce29f-cf7c-46b9-9420-1593945fa94d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:35.791335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f97127-18d3-40af-8e7f-7c8c0573bb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:35.792202 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f97127-18d3-40af-8e7f-7c8c0573bb5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:35.792488 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f97127-18d3-40af-8e7f-7c8c0573bb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:35.792773 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f97127-18d3-40af-8e7f-7c8c0573bb5b admin admin] PATH_INFO: `/v3/projects/9c40e009c5eb4817842f753ddb74b1c3/users/377ff13961e949ff88469796c4167534/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:35.807471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f97127-18d3-40af-8e7f-7c8c0573bb5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c40e009c5eb4817842f753ddb74b1c3, user_id=377ff13961e949ff88469796c4167534, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:35.810187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f97127-18d3-40af-8e7f-7c8c0573bb5b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:35.824410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1184/2385] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:34:35 2022] PUT /identity/v3/projects/9c40e009c5eb4817842f753ddb74b1c3/users/377ff13961e949ff88469796c4167534/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:34:35.835495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58201557-b9bc-4e19-9f96-3f19fefa589c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:35.835791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58201557-b9bc-4e19-9f96-3f19fefa589c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:35.836078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58201557-b9bc-4e19-9f96-3f19fefa589c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:35.864661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-58201557-b9bc-4e19-9f96-3f19fefa589c None None] MFA Rules not processed for user `377ff13961e949ff88469796c4167534`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:34:35.865995 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-58201557-b9bc-4e19-9f96-3f19fefa589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:35.940038 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1202/2386] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:34:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:34:35.952611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f97127-18d3-40af-8e7f-7c8c0573bb5b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:35.954385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-88f97127-18d3-40af-8e7f-7c8c0573bb5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:35.976631 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:34:36.016544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f97127-18d3-40af-8e7f-7c8c0573bb5b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:36.020845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b9f29-dd64-4219-a817-c1c7a3fa49d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:36.021684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b9f29-dd64-4219-a817-c1c7a3fa49d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:36.021877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b9f29-dd64-4219-a817-c1c7a3fa49d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:36.022182 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b9f29-dd64-4219-a817-c1c7a3fa49d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:36.032912 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b9f29-dd64-4219-a817-c1c7a3fa49d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:36.034580 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b9f29-dd64-4219-a817-c1c7a3fa49d5 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:36.045783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1185/2387] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:34:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:34:36.492150 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58201557-b9bc-4e19-9f96-3f19fefa589c None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:36.494301 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-58201557-b9bc-4e19-9f96-3f19fefa589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:36.533136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:34:36.575856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58201557-b9bc-4e19-9f96-3f19fefa589c None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:36.579982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27eda5-5e25-4da8-b7df-1747e840db3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:36.580661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27eda5-5e25-4da8-b7df-1747e840db3b service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:36.580952 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27eda5-5e25-4da8-b7df-1747e840db3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:36.581236 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27eda5-5e25-4da8-b7df-1747e840db3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:36.583150 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2d27eda5-5e25-4da8-b7df-1747e840db3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:36.651832 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27eda5-5e25-4da8-b7df-1747e840db3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:36.653883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27eda5-5e25-4da8-b7df-1747e840db3b service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:36.666565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1203/2388] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:34:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:34:46.040341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=17 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:34:46.660113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=42 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:34:46.803359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e4567-907c-4828-85b0-003068210d69 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:46.803649 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e4567-907c-4828-85b0-003068210d69 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:46.803929 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e4567-907c-4828-85b0-003068210d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:46.821595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-039e4567-907c-4828-85b0-003068210d69 None None] MFA Rules not processed for user `cb2bf8531dfe42b29b0b632b39d49625`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:34:46.822812 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-039e4567-907c-4828-85b0-003068210d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:46.884044 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1186/2389] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:34:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:34:47.801345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27eda5-5e25-4da8-b7df-1747e840db3b service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:47.878787 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27eda5-5e25-4da8-b7df-1747e840db3b service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:47.883782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d214ac-e61d-4427-82fb-39daca2c9421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:47.884908 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d214ac-e61d-4427-82fb-39daca2c9421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:47.885052 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d214ac-e61d-4427-82fb-39daca2c9421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:47.885241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d214ac-e61d-4427-82fb-39daca2c9421 admin admin] PATH_INFO: `/v3/users/377ff13961e949ff88469796c4167534` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:47.886583 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d214ac-e61d-4427-82fb-39daca2c9421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=377ff13961e949ff88469796c4167534)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:47.888758 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d214ac-e61d-4427-82fb-39daca2c9421 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:47.943264 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a5d214ac-e61d-4427-82fb-39daca2c9421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '377ff13961e949ff88469796c4167534'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:47.943927 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a5d214ac-e61d-4427-82fb-39daca2c9421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '377ff13961e949ff88469796c4167534'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:47.958250 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a5d214ac-e61d-4427-82fb-39daca2c9421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '377ff13961e949ff88469796c4167534'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:47.959007 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a5d214ac-e61d-4427-82fb-39daca2c9421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '377ff13961e949ff88469796c4167534'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:47.967456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a5d214ac-e61d-4427-82fb-39daca2c9421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '377ff13961e949ff88469796c4167534'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:47.981025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1204/2390] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:34:47 2022] DELETE /identity/v3/users/377ff13961e949ff88469796c4167534 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:34:47.986470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf45985-7f61-4773-8801-0ed312065964 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:47.987075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf45985-7f61-4773-8801-0ed312065964 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:47.987075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf45985-7f61-4773-8801-0ed312065964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:48.006762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-6cf45985-7f61-4773-8801-0ed312065964 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:34:48.008061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6cf45985-7f61-4773-8801-0ed312065964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:48.072237 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1187/2391] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:34:48.083811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d214ac-e61d-4427-82fb-39daca2c9421 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:48.087587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a5d214ac-e61d-4427-82fb-39daca2c9421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:48.148327 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d214ac-e61d-4427-82fb-39daca2c9421 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:48.152071 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6bad9-be14-4cff-b0bc-954aedd03961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:48.153037 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6bad9-be14-4cff-b0bc-954aedd03961 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:48.153354 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6bad9-be14-4cff-b0bc-954aedd03961 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:48.153642 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6bad9-be14-4cff-b0bc-954aedd03961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:48.163577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6bad9-be14-4cff-b0bc-954aedd03961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:48.164950 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6bad9-be14-4cff-b0bc-954aedd03961 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:48.175381 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1205/2392] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:34:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:34:48.449473 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf45985-7f61-4773-8801-0ed312065964 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:48.451956 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6cf45985-7f61-4773-8801-0ed312065964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:48.477368 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf45985-7f61-4773-8801-0ed312065964 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:48.488251 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c624c8-32d2-42e8-b91e-dfe64ccba423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:48.489906 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c624c8-32d2-42e8-b91e-dfe64ccba423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:48.490447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c624c8-32d2-42e8-b91e-dfe64ccba423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:48.490964 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c624c8-32d2-42e8-b91e-dfe64ccba423 admin admin] PATH_INFO: `/v3/projects/9c40e009c5eb4817842f753ddb74b1c3` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:48.493222 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c624c8-32d2-42e8-b91e-dfe64ccba423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c40e009c5eb4817842f753ddb74b1c3)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:48.497148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c624c8-32d2-42e8-b91e-dfe64ccba423 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:48.570982 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-47c624c8-32d2-42e8-b91e-dfe64ccba423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c40e009c5eb4817842f753ddb74b1c3', 'request_id': 'req-47c624c8-32d2-42e8-b91e-dfe64ccba423'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:48.580142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-47c624c8-32d2-42e8-b91e-dfe64ccba423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c40e009c5eb4817842f753ddb74b1c3', 'request_id': 'req-47c624c8-32d2-42e8-b91e-dfe64ccba423'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:48.588027 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-47c624c8-32d2-42e8-b91e-dfe64ccba423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c40e009c5eb4817842f753ddb74b1c3', 'request_id': 'req-47c624c8-32d2-42e8-b91e-dfe64ccba423'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:48.601112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-47c624c8-32d2-42e8-b91e-dfe64ccba423 admin admin] The token cache is being invalidate because project 9c40e009c5eb4817842f753ddb74b1c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:34:48.601502 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-47c624c8-32d2-42e8-b91e-dfe64ccba423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:48.602180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-47c624c8-32d2-42e8-b91e-dfe64ccba423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:48.603902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1188/2393] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:34:48 2022] DELETE /identity/v3/projects/9c40e009c5eb4817842f753ddb74b1c3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:34:56.308289 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6bad9-be14-4cff-b0bc-954aedd03961 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:56.309731 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-96c6bad9-be14-4cff-b0bc-954aedd03961 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:56.322811 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6bad9-be14-4cff-b0bc-954aedd03961 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:56.341703 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:34:56.381897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde0859-42c1-47e0-980e-482b5ec9b93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:56.382772 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde0859-42c1-47e0-980e-482b5ec9b93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:56.383055 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde0859-42c1-47e0-980e-482b5ec9b93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:56.383335 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde0859-42c1-47e0-980e-482b5ec9b93a admin admin] PATH_INFO: `/v3/users/b358dc1ebb56475290dd002f48b45f81` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:56.384520 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde0859-42c1-47e0-980e-482b5ec9b93a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b358dc1ebb56475290dd002f48b45f81)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:56.386467 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde0859-42c1-47e0-980e-482b5ec9b93a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:56.433599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9cde0859-42c1-47e0-980e-482b5ec9b93a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b358dc1ebb56475290dd002f48b45f81'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:56.434267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9cde0859-42c1-47e0-980e-482b5ec9b93a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b358dc1ebb56475290dd002f48b45f81'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:56.445941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9cde0859-42c1-47e0-980e-482b5ec9b93a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b358dc1ebb56475290dd002f48b45f81'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:56.446583 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9cde0859-42c1-47e0-980e-482b5ec9b93a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b358dc1ebb56475290dd002f48b45f81'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:56.454845 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9cde0859-42c1-47e0-980e-482b5ec9b93a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b358dc1ebb56475290dd002f48b45f81'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:56.466998 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1206/2394] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:34:56 2022] DELETE /identity/v3/users/b358dc1ebb56475290dd002f48b45f81 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:34:56.472265 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a860f-ce86-4a78-9eea-56b99ca6dd1d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:56.472727 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a860f-ce86-4a78-9eea-56b99ca6dd1d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:56.472858 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a860f-ce86-4a78-9eea-56b99ca6dd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:56.490349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2e3a860f-ce86-4a78-9eea-56b99ca6dd1d None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:34:56.491536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2e3a860f-ce86-4a78-9eea-56b99ca6dd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:56.515554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:34:56.552056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1189/2395] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:34:56.561600 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde0859-42c1-47e0-980e-482b5ec9b93a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:56.563489 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9cde0859-42c1-47e0-980e-482b5ec9b93a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:56.621053 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde0859-42c1-47e0-980e-482b5ec9b93a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:56.624800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30e2f3-0f0e-4b62-a9e7-0acea1d43bd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:56.625505 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30e2f3-0f0e-4b62-a9e7-0acea1d43bd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:56.625786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30e2f3-0f0e-4b62-a9e7-0acea1d43bd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:56.626062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30e2f3-0f0e-4b62-a9e7-0acea1d43bd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:56.635064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30e2f3-0f0e-4b62-a9e7-0acea1d43bd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:56.636502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30e2f3-0f0e-4b62-a9e7-0acea1d43bd1 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:56.645911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1207/2396] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:34:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:34:56.924898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a860f-ce86-4a78-9eea-56b99ca6dd1d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:34:56.926525 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2e3a860f-ce86-4a78-9eea-56b99ca6dd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:34:56.945095 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a860f-ce86-4a78-9eea-56b99ca6dd1d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:34:56.952244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75982e-cd20-40cc-bb38-9b9908bae42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:34:56.954132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75982e-cd20-40cc-bb38-9b9908bae42b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:34:56.954547 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75982e-cd20-40cc-bb38-9b9908bae42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:34:56.954735 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75982e-cd20-40cc-bb38-9b9908bae42b admin admin] PATH_INFO: `/v3/projects/af1d5b8b00b74c88a836613465585c42` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:34:56.956075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75982e-cd20-40cc-bb38-9b9908bae42b admin admin] RBAC: Authorizing `identity:delete_project(project_id=af1d5b8b00b74c88a836613465585c42)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:34:56.958102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75982e-cd20-40cc-bb38-9b9908bae42b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:34:57.010927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ce75982e-cd20-40cc-bb38-9b9908bae42b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af1d5b8b00b74c88a836613465585c42', 'request_id': 'req-ce75982e-cd20-40cc-bb38-9b9908bae42b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:57.020039 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ce75982e-cd20-40cc-bb38-9b9908bae42b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af1d5b8b00b74c88a836613465585c42', 'request_id': 'req-ce75982e-cd20-40cc-bb38-9b9908bae42b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:57.026278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ce75982e-cd20-40cc-bb38-9b9908bae42b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af1d5b8b00b74c88a836613465585c42', 'request_id': 'req-ce75982e-cd20-40cc-bb38-9b9908bae42b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:57.039504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ce75982e-cd20-40cc-bb38-9b9908bae42b admin admin] The token cache is being invalidate because project af1d5b8b00b74c88a836613465585c42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:34:57.039794 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ce75982e-cd20-40cc-bb38-9b9908bae42b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:57.040470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-ce75982e-cd20-40cc-bb38-9b9908bae42b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:34:57.042105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1190/2397] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:34:56 2022] DELETE /identity/v3/projects/af1d5b8b00b74c88a836613465585c42 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:35:06.641756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=69,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:35:07.036442 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:35:35.402958 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1251b6-975b-4888-9f66-647e41dc050d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:35:35.403671 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1251b6-975b-4888-9f66-647e41dc050d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:35:35.403671 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1251b6-975b-4888-9f66-647e41dc050d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:35:35.423318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-fa1251b6-975b-4888-9f66-647e41dc050d None None] MFA Rules not processed for user `6a71c3c016564c3b9ab0b6b58d378fda`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:35:35.424655 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fa1251b6-975b-4888-9f66-647e41dc050d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:35:35.488482 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1208/2398] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:35:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:35:37.066239 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75982e-cd20-40cc-bb38-9b9908bae42b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:35:37.067689 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ce75982e-cd20-40cc-bb38-9b9908bae42b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:35:37.137497 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75982e-cd20-40cc-bb38-9b9908bae42b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:35:37.142089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f80532-2e43-4696-b404-4802cf8f6c98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:35:37.142791 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f80532-2e43-4696-b404-4802cf8f6c98 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:35:37.143081 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f80532-2e43-4696-b404-4802cf8f6c98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:35:37.143575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f80532-2e43-4696-b404-4802cf8f6c98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:35:37.145992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-11f80532-2e43-4696-b404-4802cf8f6c98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:35:37.252363 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f80532-2e43-4696-b404-4802cf8f6c98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:35:37.253870 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f80532-2e43-4696-b404-4802cf8f6c98 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:35:37.278060 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1191/2399] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:35:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:35:38.460619 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1251b6-975b-4888-9f66-647e41dc050d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:35:38.462454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fa1251b6-975b-4888-9f66-647e41dc050d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:35:38.476452 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1251b6-975b-4888-9f66-647e41dc050d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:35:38.480157 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0b3fe-0090-46e4-ae75-59d4025fd0ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:35:38.480822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0b3fe-0090-46e4-ae75-59d4025fd0ab service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:35:38.481098 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0b3fe-0090-46e4-ae75-59d4025fd0ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:35:38.481406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0b3fe-0090-46e4-ae75-59d4025fd0ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:35:38.483809 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-18d0b3fe-0090-46e4-ae75-59d4025fd0ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:35:38.499526 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0b3fe-0090-46e4-ae75-59d4025fd0ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:35:38.500996 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0b3fe-0090-46e4-ae75-59d4025fd0ab service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:35:38.513164 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1209/2400] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:35:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:35:47.275448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=63 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:35:48.507789 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=31 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:35:58.003274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4f8df-1916-49be-b161-f3f99abd1663 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:35:58.003841 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4f8df-1916-49be-b161-f3f99abd1663 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:35:58.004604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4f8df-1916-49be-b161-f3f99abd1663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:35:58.023379 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-82e4f8df-1916-49be-b161-f3f99abd1663 None None] MFA Rules not processed for user `7a3536012d1444a1b9b606c9ec94e5cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:35:58.024653 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-82e4f8df-1916-49be-b161-f3f99abd1663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:35:58.094869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1192/2401] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:35:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:35:58.503958 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0b3fe-0090-46e4-ae75-59d4025fd0ab service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:35:58.514583 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0b3fe-0090-46e4-ae75-59d4025fd0ab service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:35:58.518953 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9b462-a281-455b-972f-49891523714f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:35:58.519647 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9b462-a281-455b-972f-49891523714f service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:35:58.519930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9b462-a281-455b-972f-49891523714f service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:35:58.520215 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9b462-a281-455b-972f-49891523714f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:35:58.522492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1ab9b462-a281-455b-972f-49891523714f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:35:58.541188 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9b462-a281-455b-972f-49891523714f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:35:58.544165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9b462-a281-455b-972f-49891523714f service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:35:58.557678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1210/2402] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:35:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:35:59.145735 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4f8df-1916-49be-b161-f3f99abd1663 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:35:59.148459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-82e4f8df-1916-49be-b161-f3f99abd1663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:35:59.239578 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4f8df-1916-49be-b161-f3f99abd1663 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:35:59.245824 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c56094-e6ff-4cf2-bc09-578144c1f841 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b76876d821e4bbbb8d5cad5bd26f559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:35:59.246511 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c56094-e6ff-4cf2-bc09-578144c1f841 service placement] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:35:59.246794 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c56094-e6ff-4cf2-bc09-578144c1f841 service placement] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:35:59.247296 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c56094-e6ff-4cf2-bc09-578144c1f841 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:35:59.249830 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-13c56094-e6ff-4cf2-bc09-578144c1f841 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:35:59.264019 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c56094-e6ff-4cf2-bc09-578144c1f841 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:35:59.265983 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c56094-e6ff-4cf2-bc09-578144c1f841 service placement] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:35:59.278715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1193/2403] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:35:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:36:01.989050 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9b462-a281-455b-972f-49891523714f service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:02.003965 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9b462-a281-455b-972f-49891523714f service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:02.008434 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42571b90-9265-44ed-b881-49c10c2fd684 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:02.009219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42571b90-9265-44ed-b881-49c10c2fd684 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:02.009606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42571b90-9265-44ed-b881-49c10c2fd684 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:02.009950 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42571b90-9265-44ed-b881-49c10c2fd684 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:02.012635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-42571b90-9265-44ed-b881-49c10c2fd684 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:02.024494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42571b90-9265-44ed-b881-49c10c2fd684 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:02.026039 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42571b90-9265-44ed-b881-49c10c2fd684 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:02.037257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1211/2404] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:36:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:36:04.522725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e8e1b-cbdf-46d3-9815-4eca2fcc57e4 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:04.523254 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e8e1b-cbdf-46d3-9815-4eca2fcc57e4 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:04.523371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e8e1b-cbdf-46d3-9815-4eca2fcc57e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:04.542407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-562e8e1b-cbdf-46d3-9815-4eca2fcc57e4 None None] MFA Rules not processed for user `cb2bf8531dfe42b29b0b632b39d49625`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:36:04.543621 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-562e8e1b-cbdf-46d3-9815-4eca2fcc57e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:04.602306 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1194/2405] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:36:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:36:07.620796 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42571b90-9265-44ed-b881-49c10c2fd684 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:07.623881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-42571b90-9265-44ed-b881-49c10c2fd684 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:07.646480 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42571b90-9265-44ed-b881-49c10c2fd684 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:07.655258 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb31420-8e93-4e08-8522-373f44b9e5a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:07.656257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb31420-8e93-4e08-8522-373f44b9e5a9 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:07.656860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb31420-8e93-4e08-8522-373f44b9e5a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:07.657411 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb31420-8e93-4e08-8522-373f44b9e5a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:07.662034 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-afb31420-8e93-4e08-8522-373f44b9e5a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:07.676240 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb31420-8e93-4e08-8522-373f44b9e5a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:07.678431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb31420-8e93-4e08-8522-373f44b9e5a9 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:07.692538 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1212/2406] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:36:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:36:08.501680 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e8e1b-cbdf-46d3-9815-4eca2fcc57e4 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:08.524522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e8e1b-cbdf-46d3-9815-4eca2fcc57e4 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:08.536647 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9deb9d-6ec9-4042-83a2-e41fb38a35c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:08.537859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9deb9d-6ec9-4042-83a2-e41fb38a35c9 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:08.538366 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9deb9d-6ec9-4042-83a2-e41fb38a35c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:08.538872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9deb9d-6ec9-4042-83a2-e41fb38a35c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:08.542283 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3c9deb9d-6ec9-4042-83a2-e41fb38a35c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:08.568105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9deb9d-6ec9-4042-83a2-e41fb38a35c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:08.570004 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9deb9d-6ec9-4042-83a2-e41fb38a35c9 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:08.587492 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1195/2407] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:36:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:36:09.949124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb31420-8e93-4e08-8522-373f44b9e5a9 service nova] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:09.951170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-afb31420-8e93-4e08-8522-373f44b9e5a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:09.968397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb31420-8e93-4e08-8522-373f44b9e5a9 service nova] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:09.974351 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb405db1-35d5-4581-85bc-232470155175 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:09.975123 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb405db1-35d5-4581-85bc-232470155175 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:09.975421 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb405db1-35d5-4581-85bc-232470155175 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:09.975770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb405db1-35d5-4581-85bc-232470155175 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:09.977727 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bb405db1-35d5-4581-85bc-232470155175 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:09.990130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb405db1-35d5-4581-85bc-232470155175 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:09.991783 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb405db1-35d5-4581-85bc-232470155175 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:10.002700 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1213/2408] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:36:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:36:18.582030 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=88 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:36:18.861047 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9deb9d-6ec9-4042-83a2-e41fb38a35c9 service nova] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:18.862659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3c9deb9d-6ec9-4042-83a2-e41fb38a35c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:18.924800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9deb9d-6ec9-4042-83a2-e41fb38a35c9 service nova] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:18.929157 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e14b1c-8e4c-4140-b9ff-a3daa9ce7207 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:18.930195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e14b1c-8e4c-4140-b9ff-a3daa9ce7207 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:18.930609 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e14b1c-8e4c-4140-b9ff-a3daa9ce7207 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:18.931010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e14b1c-8e4c-4140-b9ff-a3daa9ce7207 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:18.933602 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-34e14b1c-8e4c-4140-b9ff-a3daa9ce7207 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:18.946538 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e14b1c-8e4c-4140-b9ff-a3daa9ce7207 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:18.948161 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e14b1c-8e4c-4140-b9ff-a3daa9ce7207 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:18.964054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1196/2409] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:36:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:36:19.997267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=24 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:36:21.265878 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dcaa2f-ced1-4c55-88a4-6b6a0cc9fe24 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:21.266490 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dcaa2f-ced1-4c55-88a4-6b6a0cc9fe24 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:21.266650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dcaa2f-ced1-4c55-88a4-6b6a0cc9fe24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:21.287238 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-92dcaa2f-ced1-4c55-88a4-6b6a0cc9fe24 None None] MFA Rules not processed for user `aee77a3ed91445ab8b7f58e7f8bc348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:36:21.288496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-92dcaa2f-ced1-4c55-88a4-6b6a0cc9fe24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:21.367182 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1214/2410] 10.222.0.65 () {58 vars in 967 bytes} [Thu May 19 06:36:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:36:21.419555 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e14b1c-8e4c-4140-b9ff-a3daa9ce7207 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:21.421048 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-34e14b1c-8e4c-4140-b9ff-a3daa9ce7207 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:21.493572 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e14b1c-8e4c-4140-b9ff-a3daa9ce7207 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:21.499836 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9681ed6-65ac-429c-94ec-5070f60a733b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc205d053d54506bb6c9227e4a1b044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:21.500508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9681ed6-65ac-429c-94ec-5070f60a733b service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:21.500799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9681ed6-65ac-429c-94ec-5070f60a733b service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:21.501080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9681ed6-65ac-429c-94ec-5070f60a733b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:21.510139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9681ed6-65ac-429c-94ec-5070f60a733b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:21.511785 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9681ed6-65ac-429c-94ec-5070f60a733b service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:21.521327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1197/2411] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 06:36:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 06:36:31.360338 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=25 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:36:31.518223 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=58 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:36:32.202274 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dcaa2f-ced1-4c55-88a4-6b6a0cc9fe24 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:32.203789 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-92dcaa2f-ced1-4c55-88a4-6b6a0cc9fe24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:32.287632 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dcaa2f-ced1-4c55-88a4-6b6a0cc9fe24 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:32.292640 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1373593-22cb-4109-b772-35cebfd244e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:32.293647 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1373593-22cb-4109-b772-35cebfd244e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:32.293932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1373593-22cb-4109-b772-35cebfd244e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:32.294226 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1373593-22cb-4109-b772-35cebfd244e8 admin admin] PATH_INFO: `/v3/users/d4008c6bb57b4a91b2761fc87a1e9d2f` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:32.295683 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1373593-22cb-4109-b772-35cebfd244e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4008c6bb57b4a91b2761fc87a1e9d2f)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:32.297606 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1373593-22cb-4109-b772-35cebfd244e8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:32.370384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f1373593-22cb-4109-b772-35cebfd244e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4008c6bb57b4a91b2761fc87a1e9d2f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:32.373006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f1373593-22cb-4109-b772-35cebfd244e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4008c6bb57b4a91b2761fc87a1e9d2f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:32.396885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f1373593-22cb-4109-b772-35cebfd244e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4008c6bb57b4a91b2761fc87a1e9d2f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:32.398400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f1373593-22cb-4109-b772-35cebfd244e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4008c6bb57b4a91b2761fc87a1e9d2f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:32.411499 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f1373593-22cb-4109-b772-35cebfd244e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4008c6bb57b4a91b2761fc87a1e9d2f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:32.428911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1215/2412] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:36:32 2022] DELETE /identity/v3/users/d4008c6bb57b4a91b2761fc87a1e9d2f => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:36:32.649096 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9681ed6-65ac-429c-94ec-5070f60a733b service swift] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:32.671475 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9681ed6-65ac-429c-94ec-5070f60a733b service swift] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:32.747584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91062d6-16cd-4777-bc1f-1c0ba12519ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:32.748481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91062d6-16cd-4777-bc1f-1c0ba12519ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:32.748768 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91062d6-16cd-4777-bc1f-1c0ba12519ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:32.749050 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91062d6-16cd-4777-bc1f-1c0ba12519ea admin admin] PATH_INFO: `/v3/projects/14bd96bd91c44b888074b760e9834814` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:32.750389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91062d6-16cd-4777-bc1f-1c0ba12519ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=14bd96bd91c44b888074b760e9834814)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:32.752382 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91062d6-16cd-4777-bc1f-1c0ba12519ea admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:32.809981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c91062d6-16cd-4777-bc1f-1c0ba12519ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14bd96bd91c44b888074b760e9834814', 'request_id': 'req-c91062d6-16cd-4777-bc1f-1c0ba12519ea'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:32.819842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c91062d6-16cd-4777-bc1f-1c0ba12519ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14bd96bd91c44b888074b760e9834814', 'request_id': 'req-c91062d6-16cd-4777-bc1f-1c0ba12519ea'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:32.831454 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c91062d6-16cd-4777-bc1f-1c0ba12519ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14bd96bd91c44b888074b760e9834814', 'request_id': 'req-c91062d6-16cd-4777-bc1f-1c0ba12519ea'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:32.850305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c91062d6-16cd-4777-bc1f-1c0ba12519ea admin admin] The token cache is being invalidate because project 14bd96bd91c44b888074b760e9834814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:36:32.850603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c91062d6-16cd-4777-bc1f-1c0ba12519ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:32.851508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c91062d6-16cd-4777-bc1f-1c0ba12519ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:32.853100 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1198/2413] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:36:32 2022] DELETE /identity/v3/projects/14bd96bd91c44b888074b760e9834814 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:36:32.858075 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1373593-22cb-4109-b772-35cebfd244e8 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:32.859706 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f1373593-22cb-4109-b772-35cebfd244e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:32.877465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1373593-22cb-4109-b772-35cebfd244e8 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:32.953032 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae40c97-bb4e-43e9-926e-22d7c94f6734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:32.956149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae40c97-bb4e-43e9-926e-22d7c94f6734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:32.956149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae40c97-bb4e-43e9-926e-22d7c94f6734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:32.956149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae40c97-bb4e-43e9-926e-22d7c94f6734 admin admin] PATH_INFO: `/v3/users/75ae2c185a764de1864f87a1d109791d` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:32.956713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae40c97-bb4e-43e9-926e-22d7c94f6734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75ae2c185a764de1864f87a1d109791d)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:32.960496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae40c97-bb4e-43e9-926e-22d7c94f6734 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:32.994100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:36:33.030478 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4ae40c97-bb4e-43e9-926e-22d7c94f6734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75ae2c185a764de1864f87a1d109791d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:33.031183 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4ae40c97-bb4e-43e9-926e-22d7c94f6734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75ae2c185a764de1864f87a1d109791d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:33.046215 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4ae40c97-bb4e-43e9-926e-22d7c94f6734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75ae2c185a764de1864f87a1d109791d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:33.046957 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4ae40c97-bb4e-43e9-926e-22d7c94f6734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75ae2c185a764de1864f87a1d109791d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:33.057552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4ae40c97-bb4e-43e9-926e-22d7c94f6734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75ae2c185a764de1864f87a1d109791d'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:33.076730 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1216/2414] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:36:32 2022] DELETE /identity/v3/users/75ae2c185a764de1864f87a1d109791d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:36:33.295723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91062d6-16cd-4777-bc1f-1c0ba12519ea admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:33.297641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c91062d6-16cd-4777-bc1f-1c0ba12519ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:33.314960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91062d6-16cd-4777-bc1f-1c0ba12519ea admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:33.422944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33f48e-a355-426b-9cd5-2f7252284893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:33.424094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33f48e-a355-426b-9cd5-2f7252284893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:33.424455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33f48e-a355-426b-9cd5-2f7252284893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:33.424958 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33f48e-a355-426b-9cd5-2f7252284893 admin admin] PATH_INFO: `/v3/projects/f830b7e66df64a10aab07af6820ef125` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:33.426857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33f48e-a355-426b-9cd5-2f7252284893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f830b7e66df64a10aab07af6820ef125)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:33.429891 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33f48e-a355-426b-9cd5-2f7252284893 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:33.488872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:36:33.514195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4f33f48e-a355-426b-9cd5-2f7252284893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f830b7e66df64a10aab07af6820ef125', 'request_id': 'req-4f33f48e-a355-426b-9cd5-2f7252284893'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:33.543254 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4f33f48e-a355-426b-9cd5-2f7252284893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f830b7e66df64a10aab07af6820ef125', 'request_id': 'req-4f33f48e-a355-426b-9cd5-2f7252284893'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:33.566057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4f33f48e-a355-426b-9cd5-2f7252284893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f830b7e66df64a10aab07af6820ef125', 'request_id': 'req-4f33f48e-a355-426b-9cd5-2f7252284893'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:33.592222 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4f33f48e-a355-426b-9cd5-2f7252284893 admin admin] The token cache is being invalidate because project f830b7e66df64a10aab07af6820ef125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:36:33.592835 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4f33f48e-a355-426b-9cd5-2f7252284893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:33.594090 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4f33f48e-a355-426b-9cd5-2f7252284893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:33.596347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1199/2415] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:36:33 2022] DELETE /identity/v3/projects/f830b7e66df64a10aab07af6820ef125 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:36:41.251589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae40c97-bb4e-43e9-926e-22d7c94f6734 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:41.253093 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4ae40c97-bb4e-43e9-926e-22d7c94f6734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:41.326525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae40c97-bb4e-43e9-926e-22d7c94f6734 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:41.395041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d6f32-97ec-45df-916c-78fd839f280c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:41.395906 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d6f32-97ec-45df-916c-78fd839f280c service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:41.396266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d6f32-97ec-45df-916c-78fd839f280c service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:41.396587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d6f32-97ec-45df-916c-78fd839f280c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:41.399089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f71d6f32-97ec-45df-916c-78fd839f280c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:41.482860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d6f32-97ec-45df-916c-78fd839f280c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:41.484352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d6f32-97ec-45df-916c-78fd839f280c service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:41.495529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1217/2416] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:36:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:36:43.587751 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:36:44.983755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33f48e-a355-426b-9cd5-2f7252284893 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:44.985868 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4f33f48e-a355-426b-9cd5-2f7252284893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:45.001774 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33f48e-a355-426b-9cd5-2f7252284893 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:45.006888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a77a9-0146-4a8a-8f2d-2a3ff4f1572c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:45.007785 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a77a9-0146-4a8a-8f2d-2a3ff4f1572c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:45.008075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a77a9-0146-4a8a-8f2d-2a3ff4f1572c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:45.008389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a77a9-0146-4a8a-8f2d-2a3ff4f1572c admin admin] PATH_INFO: `/v3/users/6a71c3c016564c3b9ab0b6b58d378fda` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:45.009673 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a77a9-0146-4a8a-8f2d-2a3ff4f1572c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a71c3c016564c3b9ab0b6b58d378fda)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:45.012284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a77a9-0146-4a8a-8f2d-2a3ff4f1572c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:45.078498 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-118a77a9-0146-4a8a-8f2d-2a3ff4f1572c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a71c3c016564c3b9ab0b6b58d378fda'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:45.091822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-118a77a9-0146-4a8a-8f2d-2a3ff4f1572c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a71c3c016564c3b9ab0b6b58d378fda'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:45.099760 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-118a77a9-0146-4a8a-8f2d-2a3ff4f1572c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a71c3c016564c3b9ab0b6b58d378fda'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:45.100375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-118a77a9-0146-4a8a-8f2d-2a3ff4f1572c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a71c3c016564c3b9ab0b6b58d378fda'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:45.100981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-118a77a9-0146-4a8a-8f2d-2a3ff4f1572c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a71c3c016564c3b9ab0b6b58d378fda'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:45.118765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1200/2417] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:36:44 2022] DELETE /identity/v3/users/6a71c3c016564c3b9ab0b6b58d378fda => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:36:45.310191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d6f32-97ec-45df-916c-78fd839f280c service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:45.323137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:36:45.342545 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d6f32-97ec-45df-916c-78fd839f280c service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:45.422496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310361fb-f9a6-4cf4-a39a-f688b3776556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:45.423375 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310361fb-f9a6-4cf4-a39a-f688b3776556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:45.423742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310361fb-f9a6-4cf4-a39a-f688b3776556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:45.424020 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310361fb-f9a6-4cf4-a39a-f688b3776556 admin admin] PATH_INFO: `/v3/projects/35cb15f2075c41889b91291b645170a5` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:45.425322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310361fb-f9a6-4cf4-a39a-f688b3776556 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35cb15f2075c41889b91291b645170a5)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:45.427343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310361fb-f9a6-4cf4-a39a-f688b3776556 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:45.483717 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-310361fb-f9a6-4cf4-a39a-f688b3776556 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35cb15f2075c41889b91291b645170a5', 'request_id': 'req-310361fb-f9a6-4cf4-a39a-f688b3776556'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:45.491083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-310361fb-f9a6-4cf4-a39a-f688b3776556 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35cb15f2075c41889b91291b645170a5', 'request_id': 'req-310361fb-f9a6-4cf4-a39a-f688b3776556'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:45.500517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-310361fb-f9a6-4cf4-a39a-f688b3776556 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35cb15f2075c41889b91291b645170a5', 'request_id': 'req-310361fb-f9a6-4cf4-a39a-f688b3776556'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:45.514352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-310361fb-f9a6-4cf4-a39a-f688b3776556 admin admin] The token cache is being invalidate because project 35cb15f2075c41889b91291b645170a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:36:45.514648 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-310361fb-f9a6-4cf4-a39a-f688b3776556 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:45.515350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-310361fb-f9a6-4cf4-a39a-f688b3776556 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:36:45.516948 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1218/2418] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:36:45 2022] DELETE /identity/v3/projects/35cb15f2075c41889b91291b645170a5 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:36:45.522396 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162a280-c210-4f53-98f8-6d8d22dae85d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:45.522689 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162a280-c210-4f53-98f8-6d8d22dae85d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:45.522973 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162a280-c210-4f53-98f8-6d8d22dae85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:45.551313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e162a280-c210-4f53-98f8-6d8d22dae85d None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:36:45.553642 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e162a280-c210-4f53-98f8-6d8d22dae85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:45.663558 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1201/2419] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:36:45.674970 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38341953-c5be-45fa-8ecc-00720a6ab1ca None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:45.675300 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38341953-c5be-45fa-8ecc-00720a6ab1ca None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:45.675746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38341953-c5be-45fa-8ecc-00720a6ab1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:45.708382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-38341953-c5be-45fa-8ecc-00720a6ab1ca None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:36:45.710333 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-38341953-c5be-45fa-8ecc-00720a6ab1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:45.731427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1219/2420] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:36:45.736280 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162a280-c210-4f53-98f8-6d8d22dae85d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:45.747468 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162a280-c210-4f53-98f8-6d8d22dae85d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:45.752972 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edaac00-e8f8-48f7-ad46-7c6b0a95137d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:45.753718 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edaac00-e8f8-48f7-ad46-7c6b0a95137d admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:45.753989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edaac00-e8f8-48f7-ad46-7c6b0a95137d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:45.754267 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edaac00-e8f8-48f7-ad46-7c6b0a95137d admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:45.755015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edaac00-e8f8-48f7-ad46-7c6b0a95137d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:36:45.755345 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edaac00-e8f8-48f7-ad46-7c6b0a95137d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:45.756995 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edaac00-e8f8-48f7-ad46-7c6b0a95137d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:45.770946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1202/2421] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:36:45.775575 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38341953-c5be-45fa-8ecc-00720a6ab1ca None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:45.787656 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38341953-c5be-45fa-8ecc-00720a6ab1ca None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:45.792598 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7614b-4444-4cda-9aa7-3c309feed4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:45.793949 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7614b-4444-4cda-9aa7-3c309feed4d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:45.794241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7614b-4444-4cda-9aa7-3c309feed4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:45.794528 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7614b-4444-4cda-9aa7-3c309feed4d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:45.795262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7614b-4444-4cda-9aa7-3c309feed4d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:45.797260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7614b-4444-4cda-9aa7-3c309feed4d6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:45.830730 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1220/2422] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:36:45 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:36:45.835817 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edaac00-e8f8-48f7-ad46-7c6b0a95137d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:45.848030 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edaac00-e8f8-48f7-ad46-7c6b0a95137d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:45.916537 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1b6b4-1895-4365-a1a7-fada2ac618ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:45.917513 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1b6b4-1895-4365-a1a7-fada2ac618ff admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:45.917971 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1b6b4-1895-4365-a1a7-fada2ac618ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:45.918458 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1b6b4-1895-4365-a1a7-fada2ac618ff admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:45.919404 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1b6b4-1895-4365-a1a7-fada2ac618ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:45.921500 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1b6b4-1895-4365-a1a7-fada2ac618ff admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:45.929199 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:36:45.948337 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1203/2423] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:36:45 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:36:45.955544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7614b-4444-4cda-9aa7-3c309feed4d6 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:45.967747 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7614b-4444-4cda-9aa7-3c309feed4d6 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:45.972645 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495055ad-71f8-4413-9863-0d9a534ab264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:45.973399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495055ad-71f8-4413-9863-0d9a534ab264 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:45.973686 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495055ad-71f8-4413-9863-0d9a534ab264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:45.973961 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495055ad-71f8-4413-9863-0d9a534ab264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:45.974702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495055ad-71f8-4413-9863-0d9a534ab264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:45.976333 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495055ad-71f8-4413-9863-0d9a534ab264 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:45.988711 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1221/2424] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:36:45.993982 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1b6b4-1895-4365-a1a7-fada2ac618ff admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:46.007507 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1b6b4-1895-4365-a1a7-fada2ac618ff admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:46.013875 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ac1ec-0cc7-45e8-9d17-0fda4db724b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:46.014546 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ac1ec-0cc7-45e8-9d17-0fda4db724b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:46.014830 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ac1ec-0cc7-45e8-9d17-0fda4db724b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:46.015116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ac1ec-0cc7-45e8-9d17-0fda4db724b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:46.015868 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ac1ec-0cc7-45e8-9d17-0fda4db724b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:46.017557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ac1ec-0cc7-45e8-9d17-0fda4db724b7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:46.030627 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1204/2425] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:36:46.035622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495055ad-71f8-4413-9863-0d9a534ab264 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:46.053641 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495055ad-71f8-4413-9863-0d9a534ab264 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:46.062180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d726ca1-def5-4da5-a8f0-2cbf3d949353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:46.063693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d726ca1-def5-4da5-a8f0-2cbf3d949353 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:46.064046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d726ca1-def5-4da5-a8f0-2cbf3d949353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:46.064332 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d726ca1-def5-4da5-a8f0-2cbf3d949353 admin admin] PATH_INFO: `/v3/projects/e21ed8d40dab40828a1fc685ca63ad92/users/3a8abbf889e149f5be25ecddb22384a5/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:46.088123 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d726ca1-def5-4da5-a8f0-2cbf3d949353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e21ed8d40dab40828a1fc685ca63ad92, user_id=3a8abbf889e149f5be25ecddb22384a5, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:46.092171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d726ca1-def5-4da5-a8f0-2cbf3d949353 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:46.110002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1222/2426] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:36:46 2022] PUT /identity/v3/projects/e21ed8d40dab40828a1fc685ca63ad92/users/3a8abbf889e149f5be25ecddb22384a5/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:36:46.115544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7a7ea-1d92-486a-bca3-22c32df4a7de None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:46.115842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7a7ea-1d92-486a-bca3-22c32df4a7de None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:46.116133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7a7ea-1d92-486a-bca3-22c32df4a7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:46.137064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-34b7a7ea-1d92-486a-bca3-22c32df4a7de None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:36:46.139177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-34b7a7ea-1d92-486a-bca3-22c32df4a7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:46.210772 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1205/2427] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:36:46.222046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d726ca1-def5-4da5-a8f0-2cbf3d949353 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:46.224278 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9d726ca1-def5-4da5-a8f0-2cbf3d949353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:46.267051 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:36:46.299588 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d726ca1-def5-4da5-a8f0-2cbf3d949353 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:46.303721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a15ce-d81b-4d4a-a2f8-db0c90d6262e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:46.304403 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a15ce-d81b-4d4a-a2f8-db0c90d6262e service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:46.304695 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a15ce-d81b-4d4a-a2f8-db0c90d6262e service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:46.304973 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a15ce-d81b-4d4a-a2f8-db0c90d6262e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:46.316366 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a15ce-d81b-4d4a-a2f8-db0c90d6262e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:46.317883 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a15ce-d81b-4d4a-a2f8-db0c90d6262e service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:46.330405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1223/2428] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:36:51.516068 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b45c1-8053-446d-862f-1c96754898e6 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:51.517043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b45c1-8053-446d-862f-1c96754898e6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:51.517431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b45c1-8053-446d-862f-1c96754898e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:51.551931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4e7b45c1-8053-446d-862f-1c96754898e6 None None] MFA Rules not processed for user `3a8abbf889e149f5be25ecddb22384a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:36:51.553165 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4e7b45c1-8053-446d-862f-1c96754898e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:51.655435 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1206/2429] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:36:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:36:51.671176 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a15ce-d81b-4d4a-a2f8-db0c90d6262e service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:51.672794 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a29a15ce-d81b-4d4a-a2f8-db0c90d6262e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:51.779803 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a15ce-d81b-4d4a-a2f8-db0c90d6262e service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:51.794955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a44d7-69dc-47a3-9573-1cb3b57fc0f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:51.795688 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a44d7-69dc-47a3-9573-1cb3b57fc0f6 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:51.796095 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a44d7-69dc-47a3-9573-1cb3b57fc0f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:51.796381 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a44d7-69dc-47a3-9573-1cb3b57fc0f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:51.806512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a44d7-69dc-47a3-9573-1cb3b57fc0f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:51.808218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a44d7-69dc-47a3-9573-1cb3b57fc0f6 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:51.818624 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1224/2430] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:36:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:36:53.297734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52f4d9-a37c-40f0-9325-6d802bb1530e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:53.298257 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52f4d9-a37c-40f0-9325-6d802bb1530e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:53.298359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52f4d9-a37c-40f0-9325-6d802bb1530e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:53.319014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4b52f4d9-a37c-40f0-9325-6d802bb1530e None None] MFA Rules not processed for user `3a8abbf889e149f5be25ecddb22384a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:36:53.321022 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4b52f4d9-a37c-40f0-9325-6d802bb1530e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:53.330813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1207/2431] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:36:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:36:53.767511 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a44d7-69dc-47a3-9573-1cb3b57fc0f6 service nova] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:36:53.779804 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a44d7-69dc-47a3-9573-1cb3b57fc0f6 service nova] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:36:53.784428 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101f400-8ddb-4561-98a4-7275611b90bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:36:53.785712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101f400-8ddb-4561-98a4-7275611b90bd service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:36:53.786063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101f400-8ddb-4561-98a4-7275611b90bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:36:53.786417 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101f400-8ddb-4561-98a4-7275611b90bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:36:53.788556 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f101f400-8ddb-4561-98a4-7275611b90bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:36:53.804523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101f400-8ddb-4561-98a4-7275611b90bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:36:53.806562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101f400-8ddb-4561-98a4-7275611b90bd service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:36:53.829687 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1225/2432] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:36:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:37:02.375603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52f4d9-a37c-40f0-9325-6d802bb1530e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:02.395536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52f4d9-a37c-40f0-9325-6d802bb1530e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:02.403654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042105c-c59e-4e79-b021-35341e6cb348 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:02.404331 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042105c-c59e-4e79-b021-35341e6cb348 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:02.404770 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042105c-c59e-4e79-b021-35341e6cb348 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:02.405094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042105c-c59e-4e79-b021-35341e6cb348 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:02.407093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a042105c-c59e-4e79-b021-35341e6cb348 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:02.423004 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042105c-c59e-4e79-b021-35341e6cb348 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:02.424592 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042105c-c59e-4e79-b021-35341e6cb348 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:02.436934 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1208/2433] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:37:03.813718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=52 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:05.310835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101f400-8ddb-4561-98a4-7275611b90bd service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:05.332410 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101f400-8ddb-4561-98a4-7275611b90bd service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:05.339912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058aa18-8fb6-4c3e-8df3-5deb971b946d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:05.340635 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058aa18-8fb6-4c3e-8df3-5deb971b946d service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:05.340918 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058aa18-8fb6-4c3e-8df3-5deb971b946d service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:05.341200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058aa18-8fb6-4c3e-8df3-5deb971b946d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:05.343261 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e058aa18-8fb6-4c3e-8df3-5deb971b946d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:05.357079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058aa18-8fb6-4c3e-8df3-5deb971b946d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:05.358757 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058aa18-8fb6-4c3e-8df3-5deb971b946d service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:05.371216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1226/2434] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:37:12.433893 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:INSERT=3,keystone:SELECT=75 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:15.365672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=6 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:23.508562 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042105c-c59e-4e79-b021-35341e6cb348 service nova] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:23.519085 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042105c-c59e-4e79-b021-35341e6cb348 service nova] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:23.522900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af3a5e-0cc8-4b24-93f1-9778df8b846b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:23.523575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af3a5e-0cc8-4b24-93f1-9778df8b846b service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:23.523864 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af3a5e-0cc8-4b24-93f1-9778df8b846b service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:23.524143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af3a5e-0cc8-4b24-93f1-9778df8b846b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:23.526430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-94af3a5e-0cc8-4b24-93f1-9778df8b846b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:23.541245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af3a5e-0cc8-4b24-93f1-9778df8b846b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:23.542901 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af3a5e-0cc8-4b24-93f1-9778df8b846b service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:23.554724 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1209/2435] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:37:25.126578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058aa18-8fb6-4c3e-8df3-5deb971b946d service nova] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:25.128956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e058aa18-8fb6-4c3e-8df3-5deb971b946d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:25.149387 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058aa18-8fb6-4c3e-8df3-5deb971b946d service nova] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:25.156622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486bba5-eec1-4b19-a175-e3e381ab1469 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:25.157882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486bba5-eec1-4b19-a175-e3e381ab1469 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:25.158549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486bba5-eec1-4b19-a175-e3e381ab1469 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:25.159130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486bba5-eec1-4b19-a175-e3e381ab1469 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:25.162822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3486bba5-eec1-4b19-a175-e3e381ab1469 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:25.187868 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486bba5-eec1-4b19-a175-e3e381ab1469 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:25.190909 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486bba5-eec1-4b19-a175-e3e381ab1469 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:25.210278 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1227/2436] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:37:27.622542 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af3a5e-0cc8-4b24-93f1-9778df8b846b service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:27.624843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-94af3a5e-0cc8-4b24-93f1-9778df8b846b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:27.668237 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af3a5e-0cc8-4b24-93f1-9778df8b846b service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:27.681177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9df40-21bf-48f6-80e9-b7802ee81dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:27.682889 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9df40-21bf-48f6-80e9-b7802ee81dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:27.683092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9df40-21bf-48f6-80e9-b7802ee81dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:27.683319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9df40-21bf-48f6-80e9-b7802ee81dce admin admin] PATH_INFO: `/v3/users/cb2bf8531dfe42b29b0b632b39d49625` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:27.686733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9df40-21bf-48f6-80e9-b7802ee81dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb2bf8531dfe42b29b0b632b39d49625)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:27.691600 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9df40-21bf-48f6-80e9-b7802ee81dce admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:27.790005 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2af9df40-21bf-48f6-80e9-b7802ee81dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb2bf8531dfe42b29b0b632b39d49625'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:27.809085 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2af9df40-21bf-48f6-80e9-b7802ee81dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb2bf8531dfe42b29b0b632b39d49625'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:27.819927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2af9df40-21bf-48f6-80e9-b7802ee81dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb2bf8531dfe42b29b0b632b39d49625'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:27.820720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2af9df40-21bf-48f6-80e9-b7802ee81dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb2bf8531dfe42b29b0b632b39d49625'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:27.821466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2af9df40-21bf-48f6-80e9-b7802ee81dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb2bf8531dfe42b29b0b632b39d49625'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:27.837654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1210/2437] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:37:27 2022] DELETE /identity/v3/users/cb2bf8531dfe42b29b0b632b39d49625 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:28.112738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486bba5-eec1-4b19-a175-e3e381ab1469 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:28.136528 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486bba5-eec1-4b19-a175-e3e381ab1469 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:28.264340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c631d5e-0b46-4374-a89c-d2e829a22739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:28.266163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c631d5e-0b46-4374-a89c-d2e829a22739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:28.266916 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c631d5e-0b46-4374-a89c-d2e829a22739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:28.267458 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c631d5e-0b46-4374-a89c-d2e829a22739 admin admin] PATH_INFO: `/v3/projects/adbcff8654844c22a906a2e91ebca3bb` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:28.269879 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c631d5e-0b46-4374-a89c-d2e829a22739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adbcff8654844c22a906a2e91ebca3bb)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:28.273798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c631d5e-0b46-4374-a89c-d2e829a22739 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:28.374674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5c631d5e-0b46-4374-a89c-d2e829a22739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adbcff8654844c22a906a2e91ebca3bb', 'request_id': 'req-5c631d5e-0b46-4374-a89c-d2e829a22739'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:28.384246 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5c631d5e-0b46-4374-a89c-d2e829a22739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adbcff8654844c22a906a2e91ebca3bb', 'request_id': 'req-5c631d5e-0b46-4374-a89c-d2e829a22739'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:28.395473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5c631d5e-0b46-4374-a89c-d2e829a22739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adbcff8654844c22a906a2e91ebca3bb', 'request_id': 'req-5c631d5e-0b46-4374-a89c-d2e829a22739'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:28.416211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5c631d5e-0b46-4374-a89c-d2e829a22739 admin admin] The token cache is being invalidate because project adbcff8654844c22a906a2e91ebca3bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:37:28.417263 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5c631d5e-0b46-4374-a89c-d2e829a22739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:28.418473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5c631d5e-0b46-4374-a89c-d2e829a22739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:28.420504 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1228/2438] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:37:28 2022] DELETE /identity/v3/projects/adbcff8654844c22a906a2e91ebca3bb => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:28.426160 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c901545-1522-4f4e-bc41-f539fafd350b None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:28.426580 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c901545-1522-4f4e-bc41-f539fafd350b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:28.427024 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c901545-1522-4f4e-bc41-f539fafd350b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:28.449888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1c901545-1522-4f4e-bc41-f539fafd350b None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:28.452270 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1c901545-1522-4f4e-bc41-f539fafd350b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:28.549323 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1211/2439] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:28.563576 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fdb20-b00e-4835-81e8-51475d9fdefb None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:28.564114 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fdb20-b00e-4835-81e8-51475d9fdefb None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:28.564538 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fdb20-b00e-4835-81e8-51475d9fdefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:28.588234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-501fdb20-b00e-4835-81e8-51475d9fdefb None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:28.589964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-501fdb20-b00e-4835-81e8-51475d9fdefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:28.599309 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1229/2440] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:28.606001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c901545-1522-4f4e-bc41-f539fafd350b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:28.621054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c901545-1522-4f4e-bc41-f539fafd350b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:28.629393 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b5c6d-d79e-4caa-b827-9334fcf25949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:28.630746 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b5c6d-d79e-4caa-b827-9334fcf25949 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:28.631373 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b5c6d-d79e-4caa-b827-9334fcf25949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:28.631990 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b5c6d-d79e-4caa-b827-9334fcf25949 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:28.633459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b5c6d-d79e-4caa-b827-9334fcf25949 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:37:28.634018 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b5c6d-d79e-4caa-b827-9334fcf25949 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:28.636847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b5c6d-d79e-4caa-b827-9334fcf25949 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:28.664222 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1212/2441] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:37:28.664557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fdb20-b00e-4835-81e8-51475d9fdefb None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:28.675439 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fdb20-b00e-4835-81e8-51475d9fdefb None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:28.680287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb7ac1-a567-4eb1-b7ad-8a977ec6fd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:28.680977 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb7ac1-a567-4eb1-b7ad-8a977ec6fd90 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:28.682864 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb7ac1-a567-4eb1-b7ad-8a977ec6fd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:28.683200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb7ac1-a567-4eb1-b7ad-8a977ec6fd90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:28.684077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb7ac1-a567-4eb1-b7ad-8a977ec6fd90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:28.686130 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb7ac1-a567-4eb1-b7ad-8a977ec6fd90 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:28.729930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1230/2442] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:37:28 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:37:28.734659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b5c6d-d79e-4caa-b827-9334fcf25949 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:28.747675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b5c6d-d79e-4caa-b827-9334fcf25949 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:28.802725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:28.822676 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e4c4f-e045-4b38-8cbb-0ec94f207855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:28.823865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e4c4f-e045-4b38-8cbb-0ec94f207855 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:28.824245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e4c4f-e045-4b38-8cbb-0ec94f207855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:28.824523 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e4c4f-e045-4b38-8cbb-0ec94f207855 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:28.825259 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e4c4f-e045-4b38-8cbb-0ec94f207855 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:28.827425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e4c4f-e045-4b38-8cbb-0ec94f207855 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:28.851611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1213/2443] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:37:28 2022] POST /identity/v3/users => generated 286 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:37:28.856628 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb7ac1-a567-4eb1-b7ad-8a977ec6fd90 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:28.884787 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb7ac1-a567-4eb1-b7ad-8a977ec6fd90 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:28.897666 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e924f-df65-4641-b79a-d86eb943bdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:28.900116 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e924f-df65-4641-b79a-d86eb943bdbd admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:28.900702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e924f-df65-4641-b79a-d86eb943bdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:28.901647 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e924f-df65-4641-b79a-d86eb943bdbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:28.903164 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e924f-df65-4641-b79a-d86eb943bdbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:28.906117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e924f-df65-4641-b79a-d86eb943bdbd admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:28.936051 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1231/2444] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:28.937078 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e4c4f-e045-4b38-8cbb-0ec94f207855 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:28.949131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e4c4f-e045-4b38-8cbb-0ec94f207855 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:28.954314 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fc3be-3632-4817-91ae-a106ea989336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:28.955188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fc3be-3632-4817-91ae-a106ea989336 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:28.955585 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fc3be-3632-4817-91ae-a106ea989336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:28.955973 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fc3be-3632-4817-91ae-a106ea989336 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:28.956841 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fc3be-3632-4817-91ae-a106ea989336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:28.958707 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fc3be-3632-4817-91ae-a106ea989336 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:29.000820 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e924f-df65-4641-b79a-d86eb943bdbd admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:29.005607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1214/2445] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:29.019515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e924f-df65-4641-b79a-d86eb943bdbd admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:29.030063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa227d9e-093e-480a-8ed4-568fd2bfb6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:29.031977 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa227d9e-093e-480a-8ed4-568fd2bfb6c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:29.032407 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa227d9e-093e-480a-8ed4-568fd2bfb6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:29.032936 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa227d9e-093e-480a-8ed4-568fd2bfb6c3 admin admin] PATH_INFO: `/v3/projects/f73ed51a759a40d89ce18ba2a5196ca1/users/1ed52c55f5714fb79d669d47ec55b2fd/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:29.065026 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa227d9e-093e-480a-8ed4-568fd2bfb6c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f73ed51a759a40d89ce18ba2a5196ca1, user_id=1ed52c55f5714fb79d669d47ec55b2fd, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:29.068265 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa227d9e-093e-480a-8ed4-568fd2bfb6c3 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:29.099502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1232/2446] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:28 2022] PUT /identity/v3/projects/f73ed51a759a40d89ce18ba2a5196ca1/users/1ed52c55f5714fb79d669d47ec55b2fd/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:29.108802 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddb64c-2eec-4f5a-8ce4-c55c46ddd392 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:29.109362 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddb64c-2eec-4f5a-8ce4-c55c46ddd392 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:29.109869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddb64c-2eec-4f5a-8ce4-c55c46ddd392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:29.155811 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a8ddb64c-2eec-4f5a-8ce4-c55c46ddd392 None None] MFA Rules not processed for user `1ed52c55f5714fb79d669d47ec55b2fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:29.157748 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a8ddb64c-2eec-4f5a-8ce4-c55c46ddd392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:29.241859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1215/2447] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:37:29.247754 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa227d9e-093e-480a-8ed4-568fd2bfb6c3 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:29.254001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:29.332032 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa227d9e-093e-480a-8ed4-568fd2bfb6c3 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:29.337241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703452a-8d3c-4502-9842-148975d8144b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:29.338057 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703452a-8d3c-4502-9842-148975d8144b admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:29.339020 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703452a-8d3c-4502-9842-148975d8144b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:29.339375 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703452a-8d3c-4502-9842-148975d8144b admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:29.340209 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703452a-8d3c-4502-9842-148975d8144b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:29.342274 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703452a-8d3c-4502-9842-148975d8144b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:29.410222 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1233/2448] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:37:29 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:29.428513 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddb64c-2eec-4f5a-8ce4-c55c46ddd392 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:29.448280 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddb64c-2eec-4f5a-8ce4-c55c46ddd392 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:29.538950 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d353da6-e400-4f71-8d2d-248ac2412dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:29.539661 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d353da6-e400-4f71-8d2d-248ac2412dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:29.539976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d353da6-e400-4f71-8d2d-248ac2412dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:29.540286 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d353da6-e400-4f71-8d2d-248ac2412dc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:29.541158 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d353da6-e400-4f71-8d2d-248ac2412dc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:29.544197 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d353da6-e400-4f71-8d2d-248ac2412dc6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:29.580036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1216/2449] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:37:29 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:29.581318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703452a-8d3c-4502-9842-148975d8144b admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:29.597127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703452a-8d3c-4502-9842-148975d8144b admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:29.606706 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3aee9-7fcc-4fd1-9d2e-1b345c5257c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:29.607408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3aee9-7fcc-4fd1-9d2e-1b345c5257c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:29.607739 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3aee9-7fcc-4fd1-9d2e-1b345c5257c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:29.608025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3aee9-7fcc-4fd1-9d2e-1b345c5257c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:29.608770 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3aee9-7fcc-4fd1-9d2e-1b345c5257c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:29.610580 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3aee9-7fcc-4fd1-9d2e-1b345c5257c1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:29.625381 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1234/2450] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:37:29.630683 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d353da6-e400-4f71-8d2d-248ac2412dc6 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:29.643552 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d353da6-e400-4f71-8d2d-248ac2412dc6 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:29.648990 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d66f3-8d8f-4d23-8303-a99771b6c65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:29.650170 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d66f3-8d8f-4d23-8303-a99771b6c65e admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:29.652062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d66f3-8d8f-4d23-8303-a99771b6c65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:29.652828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d66f3-8d8f-4d23-8303-a99771b6c65e admin admin] PATH_INFO: `/v3/projects/bdcfb00a3a83442c84148513ba96d81a/users/2aab8eb124414c9a9374dac1f32d85aa/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:29.693360 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d66f3-8d8f-4d23-8303-a99771b6c65e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdcfb00a3a83442c84148513ba96d81a, user_id=2aab8eb124414c9a9374dac1f32d85aa, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:29.696851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3aee9-7fcc-4fd1-9d2e-1b345c5257c1 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:29.698782 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d66f3-8d8f-4d23-8303-a99771b6c65e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:29.701699 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e4e3aee9-7fcc-4fd1-9d2e-1b345c5257c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:29.738453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3aee9-7fcc-4fd1-9d2e-1b345c5257c1 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:29.757416 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1217/2451] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:29 2022] PUT /identity/v3/projects/bdcfb00a3a83442c84148513ba96d81a/users/2aab8eb124414c9a9374dac1f32d85aa/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:29.759868 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d66f3-8d8f-4d23-8303-a99771b6c65e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:29.809584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:29.862471 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802641b-98cf-41cd-ad68-6f715cf77b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:29.863402 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802641b-98cf-41cd-ad68-6f715cf77b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:29.863687 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802641b-98cf-41cd-ad68-6f715cf77b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:29.863993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802641b-98cf-41cd-ad68-6f715cf77b33 admin admin] PATH_INFO: `/v3/users/7a3536012d1444a1b9b606c9ec94e5cf` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:29.865431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802641b-98cf-41cd-ad68-6f715cf77b33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a3536012d1444a1b9b606c9ec94e5cf)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:29.867496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802641b-98cf-41cd-ad68-6f715cf77b33 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:29.884413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d66f3-8d8f-4d23-8303-a99771b6c65e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:29.890481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef26cc-bddd-441e-a6cf-28cc575906d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:29.891184 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef26cc-bddd-441e-a6cf-28cc575906d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:29.891472 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef26cc-bddd-441e-a6cf-28cc575906d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:29.891756 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef26cc-bddd-441e-a6cf-28cc575906d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:29.892551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef26cc-bddd-441e-a6cf-28cc575906d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:29.894313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef26cc-bddd-441e-a6cf-28cc575906d9 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:29.908635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1218/2452] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:29.914752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef26cc-bddd-441e-a6cf-28cc575906d9 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:29.935657 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef26cc-bddd-441e-a6cf-28cc575906d9 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:29.944859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3a77b-2b4e-4a8d-a22d-58545c04e563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:29.946087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3a77b-2b4e-4a8d-a22d-58545c04e563 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:29.946532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3a77b-2b4e-4a8d-a22d-58545c04e563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:29.946938 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3a77b-2b4e-4a8d-a22d-58545c04e563 admin admin] PATH_INFO: `/v3/projects/bdcfb00a3a83442c84148513ba96d81a/users/2aab8eb124414c9a9374dac1f32d85aa/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:29.950231 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f3a77b-2b4e-4a8d-a22d-58545c04e563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdcfb00a3a83442c84148513ba96d81a, user_id=2aab8eb124414c9a9374dac1f32d85aa, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:29.953443 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f3a77b-2b4e-4a8d-a22d-58545c04e563 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:29.957171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9802641b-98cf-41cd-ad68-6f715cf77b33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a3536012d1444a1b9b606c9ec94e5cf'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:29.958284 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9802641b-98cf-41cd-ad68-6f715cf77b33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a3536012d1444a1b9b606c9ec94e5cf'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:29.966924 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1219/2453] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:29 2022] PUT /identity/v3/projects/bdcfb00a3a83442c84148513ba96d81a/users/2aab8eb124414c9a9374dac1f32d85aa/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:29.978544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368f7c5-0e5b-48a6-90b4-eac931159a7e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:29.978817 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368f7c5-0e5b-48a6-90b4-eac931159a7e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:29.979263 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368f7c5-0e5b-48a6-90b4-eac931159a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:29.980433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9802641b-98cf-41cd-ad68-6f715cf77b33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a3536012d1444a1b9b606c9ec94e5cf'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:29.981560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9802641b-98cf-41cd-ad68-6f715cf77b33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a3536012d1444a1b9b606c9ec94e5cf'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:29.992925 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9802641b-98cf-41cd-ad68-6f715cf77b33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a3536012d1444a1b9b606c9ec94e5cf'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:30.007828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:30.014095 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4368f7c5-0e5b-48a6-90b4-eac931159a7e None None] MFA Rules not processed for user `2aab8eb124414c9a9374dac1f32d85aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:30.015258 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4368f7c5-0e5b-48a6-90b4-eac931159a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:30.016774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1235/2454] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:37:29 2022] DELETE /identity/v3/users/7a3536012d1444a1b9b606c9ec94e5cf => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:30.123949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1220/2455] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:29 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:30.137395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802641b-98cf-41cd-ad68-6f715cf77b33 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:30.139352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9802641b-98cf-41cd-ad68-6f715cf77b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:30.220467 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802641b-98cf-41cd-ad68-6f715cf77b33 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:30.225742 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f346ee6-d0dc-498a-8c11-fafc48cc4790 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:30.226500 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f346ee6-d0dc-498a-8c11-fafc48cc4790 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:30.226796 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f346ee6-d0dc-498a-8c11-fafc48cc4790 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:30.227227 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f346ee6-d0dc-498a-8c11-fafc48cc4790 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:30.239982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f346ee6-d0dc-498a-8c11-fafc48cc4790 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:30.241930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f346ee6-d0dc-498a-8c11-fafc48cc4790 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:30.255234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1236/2456] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:30 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:37:30.274391 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368f7c5-0e5b-48a6-90b4-eac931159a7e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:30.279539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4368f7c5-0e5b-48a6-90b4-eac931159a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:30.434586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368f7c5-0e5b-48a6-90b4-eac931159a7e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:30.441174 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbdea1-f3af-4d16-86ae-93fa5c3e88f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:30.442795 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbdea1-f3af-4d16-86ae-93fa5c3e88f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:30.443334 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbdea1-f3af-4d16-86ae-93fa5c3e88f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:30.443979 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbdea1-f3af-4d16-86ae-93fa5c3e88f3 admin admin] PATH_INFO: `/v3/projects/1e8f1e4b9cd9432b90cb00a6be4d909e` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:30.446304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dbdea1-f3af-4d16-86ae-93fa5c3e88f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e8f1e4b9cd9432b90cb00a6be4d909e)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:30.450338 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dbdea1-f3af-4d16-86ae-93fa5c3e88f3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:30.518305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:30.546717 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-73dbdea1-f3af-4d16-86ae-93fa5c3e88f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8f1e4b9cd9432b90cb00a6be4d909e', 'request_id': 'req-73dbdea1-f3af-4d16-86ae-93fa5c3e88f3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:30.562039 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-73dbdea1-f3af-4d16-86ae-93fa5c3e88f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8f1e4b9cd9432b90cb00a6be4d909e', 'request_id': 'req-73dbdea1-f3af-4d16-86ae-93fa5c3e88f3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:30.572989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-73dbdea1-f3af-4d16-86ae-93fa5c3e88f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8f1e4b9cd9432b90cb00a6be4d909e', 'request_id': 'req-73dbdea1-f3af-4d16-86ae-93fa5c3e88f3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:30.591058 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-73dbdea1-f3af-4d16-86ae-93fa5c3e88f3 admin admin] The token cache is being invalidate because project 1e8f1e4b9cd9432b90cb00a6be4d909e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:37:30.591487 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-73dbdea1-f3af-4d16-86ae-93fa5c3e88f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:30.592316 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-73dbdea1-f3af-4d16-86ae-93fa5c3e88f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:30.594399 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1221/2457] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:37:30 2022] DELETE /identity/v3/projects/1e8f1e4b9cd9432b90cb00a6be4d909e => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:30.600150 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e147c2-6bc4-45ed-94ec-31b1ea1fea91 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:30.600575 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e147c2-6bc4-45ed-94ec-31b1ea1fea91 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:30.600957 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e147c2-6bc4-45ed-94ec-31b1ea1fea91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:30.630842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-95e147c2-6bc4-45ed-94ec-31b1ea1fea91 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:30.633001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-95e147c2-6bc4-45ed-94ec-31b1ea1fea91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:30.742231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1237/2458] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:30.751430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d61bb-44ed-4e49-b51f-2680535d9467 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:30.752174 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d61bb-44ed-4e49-b51f-2680535d9467 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:30.752174 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d61bb-44ed-4e49-b51f-2680535d9467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:30.775586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4e3d61bb-44ed-4e49-b51f-2680535d9467 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:30.777442 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4e3d61bb-44ed-4e49-b51f-2680535d9467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:30.793554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1222/2459] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:30.794218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e147c2-6bc4-45ed-94ec-31b1ea1fea91 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:30.805627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e147c2-6bc4-45ed-94ec-31b1ea1fea91 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:30.812384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb801fee-804d-44ae-9c69-63ef3db3cc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:30.813337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb801fee-804d-44ae-9c69-63ef3db3cc54 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:30.813785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb801fee-804d-44ae-9c69-63ef3db3cc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:30.814136 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb801fee-804d-44ae-9c69-63ef3db3cc54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:30.815064 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb801fee-804d-44ae-9c69-63ef3db3cc54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:37:30.815511 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb801fee-804d-44ae-9c69-63ef3db3cc54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:30.817389 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb801fee-804d-44ae-9c69-63ef3db3cc54 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:30.833810 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1238/2460] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:37:30.838783 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb801fee-804d-44ae-9c69-63ef3db3cc54 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:30.839047 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d61bb-44ed-4e49-b51f-2680535d9467 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:30.841458 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4e3d61bb-44ed-4e49-b51f-2680535d9467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:30.850126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb801fee-804d-44ae-9c69-63ef3db3cc54 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:30.856751 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4acec8-a8e2-4b7d-9aed-b8cda21e0cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:30.857504 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4acec8-a8e2-4b7d-9aed-b8cda21e0cbc admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:30.857923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4acec8-a8e2-4b7d-9aed-b8cda21e0cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:30.858218 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4acec8-a8e2-4b7d-9aed-b8cda21e0cbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:30.859025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4acec8-a8e2-4b7d-9aed-b8cda21e0cbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:30.861021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4acec8-a8e2-4b7d-9aed-b8cda21e0cbc admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:30.865865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d61bb-44ed-4e49-b51f-2680535d9467 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:30.874357 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a43b1f-caa4-4e6b-ab52-3d2e928c12e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:30.875837 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a43b1f-caa4-4e6b-ab52-3d2e928c12e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:30.876352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a43b1f-caa4-4e6b-ab52-3d2e928c12e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:30.876703 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a43b1f-caa4-4e6b-ab52-3d2e928c12e5 admin admin] PATH_INFO: `/v3/users/1ed52c55f5714fb79d669d47ec55b2fd` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:30.878972 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a43b1f-caa4-4e6b-ab52-3d2e928c12e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ed52c55f5714fb79d669d47ec55b2fd)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:30.883053 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a43b1f-caa4-4e6b-ab52-3d2e928c12e5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:30.899344 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1239/2461] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:37:30 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:37:30.900638 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4acec8-a8e2-4b7d-9aed-b8cda21e0cbc admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:30.966767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-59a43b1f-caa4-4e6b-ab52-3d2e928c12e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ed52c55f5714fb79d669d47ec55b2fd'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:30.981854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-59a43b1f-caa4-4e6b-ab52-3d2e928c12e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ed52c55f5714fb79d669d47ec55b2fd'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:30.994336 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-59a43b1f-caa4-4e6b-ab52-3d2e928c12e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ed52c55f5714fb79d669d47ec55b2fd'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:30.996404 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-59a43b1f-caa4-4e6b-ab52-3d2e928c12e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ed52c55f5714fb79d669d47ec55b2fd'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:30.997110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-59a43b1f-caa4-4e6b-ab52-3d2e928c12e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ed52c55f5714fb79d669d47ec55b2fd'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:30.999303 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4acec8-a8e2-4b7d-9aed-b8cda21e0cbc admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:31.005974 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c824a7-91ba-4067-b58c-4c7a5ba67f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:31.006859 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c824a7-91ba-4067-b58c-4c7a5ba67f16 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:31.007220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c824a7-91ba-4067-b58c-4c7a5ba67f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:31.007636 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c824a7-91ba-4067-b58c-4c7a5ba67f16 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:31.008448 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c824a7-91ba-4067-b58c-4c7a5ba67f16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:31.011882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c824a7-91ba-4067-b58c-4c7a5ba67f16 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:31.022633 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1223/2462] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:37:30 2022] DELETE /identity/v3/users/1ed52c55f5714fb79d669d47ec55b2fd => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:31.024166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:31.032096 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485045c4-acd3-4275-abe3-8dc8fdfb0e1a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:31.032532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485045c4-acd3-4275-abe3-8dc8fdfb0e1a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:31.033069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485045c4-acd3-4275-abe3-8dc8fdfb0e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:31.046077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1240/2463] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:37:30 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:37:31.051501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c824a7-91ba-4067-b58c-4c7a5ba67f16 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:31.053321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-19c824a7-91ba-4067-b58c-4c7a5ba67f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:31.059441 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-485045c4-acd3-4275-abe3-8dc8fdfb0e1a None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:31.060788 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-485045c4-acd3-4275-abe3-8dc8fdfb0e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:31.068065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c824a7-91ba-4067-b58c-4c7a5ba67f16 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:31.141997 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1224/2464] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:31.149101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47099f9a-af98-4ff8-8998-004bdb2ba7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:31.149852 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47099f9a-af98-4ff8-8998-004bdb2ba7a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:31.150143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47099f9a-af98-4ff8-8998-004bdb2ba7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:31.150426 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47099f9a-af98-4ff8-8998-004bdb2ba7a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:31.152441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47099f9a-af98-4ff8-8998-004bdb2ba7a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:31.154854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47099f9a-af98-4ff8-8998-004bdb2ba7a3 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:31.155389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485045c4-acd3-4275-abe3-8dc8fdfb0e1a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:31.159487 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-485045c4-acd3-4275-abe3-8dc8fdfb0e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:31.176356 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1241/2465] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:37:31.180732 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47099f9a-af98-4ff8-8998-004bdb2ba7a3 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:31.202384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47099f9a-af98-4ff8-8998-004bdb2ba7a3 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:31.203533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:31.210860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f9242-75a3-4811-989f-6288d4295f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:31.212184 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f9242-75a3-4811-989f-6288d4295f0a admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:31.212740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f9242-75a3-4811-989f-6288d4295f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:31.213262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f9242-75a3-4811-989f-6288d4295f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:31.215026 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f9242-75a3-4811-989f-6288d4295f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:31.218278 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f9242-75a3-4811-989f-6288d4295f0a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:31.237687 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1242/2466] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:31.242772 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f9242-75a3-4811-989f-6288d4295f0a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:31.245743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485045c4-acd3-4275-abe3-8dc8fdfb0e1a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:31.250131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b595b0-6474-40ae-9ed1-447630b336e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:31.250900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b595b0-6474-40ae-9ed1-447630b336e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:31.251123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b595b0-6474-40ae-9ed1-447630b336e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:31.251408 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b595b0-6474-40ae-9ed1-447630b336e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:31.257482 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f9242-75a3-4811-989f-6288d4295f0a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:31.264326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97fb9b-c9f4-41eb-bc09-bb3181af4f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:31.265479 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97fb9b-c9f4-41eb-bc09-bb3181af4f5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:31.265898 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97fb9b-c9f4-41eb-bc09-bb3181af4f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:31.266298 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97fb9b-c9f4-41eb-bc09-bb3181af4f5c admin admin] PATH_INFO: `/v3/projects/4a0a5b597c3743b1ac55fd40bd141ac7/users/f99a5aaef1a446ec819ba675dd5d47dc/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:31.269396 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b595b0-6474-40ae-9ed1-447630b336e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:31.272395 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b595b0-6474-40ae-9ed1-447630b336e1 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:31.285704 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97fb9b-c9f4-41eb-bc09-bb3181af4f5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a0a5b597c3743b1ac55fd40bd141ac7, user_id=f99a5aaef1a446ec819ba675dd5d47dc, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:31.288389 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97fb9b-c9f4-41eb-bc09-bb3181af4f5c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:31.292728 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1225/2467] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:37:31.304882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1243/2468] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:31 2022] PUT /identity/v3/projects/4a0a5b597c3743b1ac55fd40bd141ac7/users/f99a5aaef1a446ec819ba675dd5d47dc/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:31.318158 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfcd39-f892-47d0-a8ef-c93ace442095 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:31.318545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfcd39-f892-47d0-a8ef-c93ace442095 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:31.318927 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfcd39-f892-47d0-a8ef-c93ace442095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:31.369479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-badfcd39-f892-47d0-a8ef-c93ace442095 None None] MFA Rules not processed for user `f99a5aaef1a446ec819ba675dd5d47dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:31.373772 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-badfcd39-f892-47d0-a8ef-c93ace442095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:31.499936 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1226/2469] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:31.518843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97fb9b-c9f4-41eb-bc09-bb3181af4f5c admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:31.520939 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ea97fb9b-c9f4-41eb-bc09-bb3181af4f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:31.632954 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97fb9b-c9f4-41eb-bc09-bb3181af4f5c admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:31.643300 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcabf7-2881-47a7-a8de-c38a79980379 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:31.644543 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcabf7-2881-47a7-a8de-c38a79980379 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:31.644981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcabf7-2881-47a7-a8de-c38a79980379 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:31.645488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcabf7-2881-47a7-a8de-c38a79980379 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:31.660068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcabf7-2881-47a7-a8de-c38a79980379 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:31.662051 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcabf7-2881-47a7-a8de-c38a79980379 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:31.671952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfcd39-f892-47d0-a8ef-c93ace442095 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:31.674066 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-badfcd39-f892-47d0-a8ef-c93ace442095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:31.679230 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1244/2470] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:37:31.818454 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfcd39-f892-47d0-a8ef-c93ace442095 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:31.825391 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521d74d-f732-4004-80a0-105caae1a722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:31.826447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521d74d-f732-4004-80a0-105caae1a722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:31.826747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521d74d-f732-4004-80a0-105caae1a722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:31.827030 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521d74d-f732-4004-80a0-105caae1a722 admin admin] PATH_INFO: `/v3/projects/f73ed51a759a40d89ce18ba2a5196ca1` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:31.828369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521d74d-f732-4004-80a0-105caae1a722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f73ed51a759a40d89ce18ba2a5196ca1)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:31.830596 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521d74d-f732-4004-80a0-105caae1a722 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:31.902349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9521d74d-f732-4004-80a0-105caae1a722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f73ed51a759a40d89ce18ba2a5196ca1', 'request_id': 'req-9521d74d-f732-4004-80a0-105caae1a722'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:31.912949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9521d74d-f732-4004-80a0-105caae1a722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f73ed51a759a40d89ce18ba2a5196ca1', 'request_id': 'req-9521d74d-f732-4004-80a0-105caae1a722'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:31.916499 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:31.923531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9521d74d-f732-4004-80a0-105caae1a722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f73ed51a759a40d89ce18ba2a5196ca1', 'request_id': 'req-9521d74d-f732-4004-80a0-105caae1a722'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:31.952352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9521d74d-f732-4004-80a0-105caae1a722 admin admin] The token cache is being invalidate because project f73ed51a759a40d89ce18ba2a5196ca1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:37:31.953030 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9521d74d-f732-4004-80a0-105caae1a722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:31.954169 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-9521d74d-f732-4004-80a0-105caae1a722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:31.957419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1227/2471] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:37:31 2022] DELETE /identity/v3/projects/f73ed51a759a40d89ce18ba2a5196ca1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:31.965167 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcabf7-2881-47a7-a8de-c38a79980379 service nova] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:31.989271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcabf7-2881-47a7-a8de-c38a79980379 service nova] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:32.074531 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:32.116155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b9c4b-85e1-4eba-bb05-28d32bdbe43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:32.117945 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b9c4b-85e1-4eba-bb05-28d32bdbe43f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:32.118678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b9c4b-85e1-4eba-bb05-28d32bdbe43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:32.119631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b9c4b-85e1-4eba-bb05-28d32bdbe43f admin admin] PATH_INFO: `/v3/users/2aab8eb124414c9a9374dac1f32d85aa` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:32.126671 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b9c4b-85e1-4eba-bb05-28d32bdbe43f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aab8eb124414c9a9374dac1f32d85aa)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:32.129420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b9c4b-85e1-4eba-bb05-28d32bdbe43f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:32.218100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d92b9c4b-85e1-4eba-bb05-28d32bdbe43f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aab8eb124414c9a9374dac1f32d85aa'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:32.219377 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d92b9c4b-85e1-4eba-bb05-28d32bdbe43f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aab8eb124414c9a9374dac1f32d85aa'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:32.245452 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d92b9c4b-85e1-4eba-bb05-28d32bdbe43f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aab8eb124414c9a9374dac1f32d85aa'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:32.246657 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d92b9c4b-85e1-4eba-bb05-28d32bdbe43f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aab8eb124414c9a9374dac1f32d85aa'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:32.259370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-d92b9c4b-85e1-4eba-bb05-28d32bdbe43f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aab8eb124414c9a9374dac1f32d85aa'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:32.282753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1245/2472] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:37:31 2022] DELETE /identity/v3/users/2aab8eb124414c9a9374dac1f32d85aa => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:32.625230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521d74d-f732-4004-80a0-105caae1a722 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:32.627076 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-9521d74d-f732-4004-80a0-105caae1a722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:32.641508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521d74d-f732-4004-80a0-105caae1a722 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:32.709361 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2f11c-1844-419f-8f54-53a832dd4bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:32.710273 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2f11c-1844-419f-8f54-53a832dd4bb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:32.710558 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2f11c-1844-419f-8f54-53a832dd4bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:32.710839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2f11c-1844-419f-8f54-53a832dd4bb1 admin admin] PATH_INFO: `/v3/projects/bdcfb00a3a83442c84148513ba96d81a` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:32.712073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2f11c-1844-419f-8f54-53a832dd4bb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdcfb00a3a83442c84148513ba96d81a)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:32.714560 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2f11c-1844-419f-8f54-53a832dd4bb1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:32.765688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3ed2f11c-1844-419f-8f54-53a832dd4bb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdcfb00a3a83442c84148513ba96d81a', 'request_id': 'req-3ed2f11c-1844-419f-8f54-53a832dd4bb1'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:32.774211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3ed2f11c-1844-419f-8f54-53a832dd4bb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdcfb00a3a83442c84148513ba96d81a', 'request_id': 'req-3ed2f11c-1844-419f-8f54-53a832dd4bb1'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:32.782420 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3ed2f11c-1844-419f-8f54-53a832dd4bb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdcfb00a3a83442c84148513ba96d81a', 'request_id': 'req-3ed2f11c-1844-419f-8f54-53a832dd4bb1'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:32.796196 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3ed2f11c-1844-419f-8f54-53a832dd4bb1 admin admin] The token cache is being invalidate because project bdcfb00a3a83442c84148513ba96d81a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:37:32.796484 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3ed2f11c-1844-419f-8f54-53a832dd4bb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:32.797161 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-3ed2f11c-1844-419f-8f54-53a832dd4bb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:32.798777 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1228/2473] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:37:32 2022] DELETE /identity/v3/projects/bdcfb00a3a83442c84148513ba96d81a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:32.804401 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe45d7-4db8-4f70-a0b6-7d7b5182db2e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:32.804698 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe45d7-4db8-4f70-a0b6-7d7b5182db2e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:32.804984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe45d7-4db8-4f70-a0b6-7d7b5182db2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:32.824033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-6efe45d7-4db8-4f70-a0b6-7d7b5182db2e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:32.825310 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6efe45d7-4db8-4f70-a0b6-7d7b5182db2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:32.897509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1246/2474] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:37:32.906008 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af329166-9a95-4dc2-8875-708afcd2ae7a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:32.906824 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af329166-9a95-4dc2-8875-708afcd2ae7a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:32.906989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af329166-9a95-4dc2-8875-708afcd2ae7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:32.928777 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-af329166-9a95-4dc2-8875-708afcd2ae7a None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:32.930184 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-af329166-9a95-4dc2-8875-708afcd2ae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:32.938800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1229/2475] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:32.943786 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe45d7-4db8-4f70-a0b6-7d7b5182db2e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:32.955349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe45d7-4db8-4f70-a0b6-7d7b5182db2e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:32.960423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b33de1-1c4f-4b61-8c66-3acbf0c832ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:32.961115 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b33de1-1c4f-4b61-8c66-3acbf0c832ab admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:32.961440 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b33de1-1c4f-4b61-8c66-3acbf0c832ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:32.961720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b33de1-1c4f-4b61-8c66-3acbf0c832ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:32.962476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b33de1-1c4f-4b61-8c66-3acbf0c832ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:37:32.962794 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b33de1-1c4f-4b61-8c66-3acbf0c832ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:32.965129 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b33de1-1c4f-4b61-8c66-3acbf0c832ab admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:32.978995 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1247/2476] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:37:32.983628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af329166-9a95-4dc2-8875-708afcd2ae7a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:32.993624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af329166-9a95-4dc2-8875-708afcd2ae7a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:32.998976 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43740ae2-a922-497a-b9a1-f6adc449cc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:32.999664 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43740ae2-a922-497a-b9a1-f6adc449cc0f admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:32.999950 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43740ae2-a922-497a-b9a1-f6adc449cc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:33.000227 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43740ae2-a922-497a-b9a1-f6adc449cc0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:33.000932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43740ae2-a922-497a-b9a1-f6adc449cc0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:33.002955 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43740ae2-a922-497a-b9a1-f6adc449cc0f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:33.033715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1230/2477] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:37:32 2022] POST /identity/v3/projects => generated 353 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:33.038315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b33de1-1c4f-4b61-8c66-3acbf0c832ab admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:33.050031 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b33de1-1c4f-4b61-8c66-3acbf0c832ab admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:33.107388 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:33.124621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7bb37-fc37-42d5-bde1-c60456788112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:33.125502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7bb37-fc37-42d5-bde1-c60456788112 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:33.125884 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7bb37-fc37-42d5-bde1-c60456788112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:33.126253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7bb37-fc37-42d5-bde1-c60456788112 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:33.127194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7bb37-fc37-42d5-bde1-c60456788112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:33.129442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7bb37-fc37-42d5-bde1-c60456788112 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:33.154643 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1248/2478] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:37:33 2022] POST /identity/v3/users => generated 277 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:33.160163 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43740ae2-a922-497a-b9a1-f6adc449cc0f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:33.174193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43740ae2-a922-497a-b9a1-f6adc449cc0f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:33.179634 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8acb4-5851-4b26-a50a-9a348b306a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:33.180345 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8acb4-5851-4b26-a50a-9a348b306a1d admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:33.180636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8acb4-5851-4b26-a50a-9a348b306a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:33.180930 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8acb4-5851-4b26-a50a-9a348b306a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:33.181739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8acb4-5851-4b26-a50a-9a348b306a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:33.184865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8acb4-5851-4b26-a50a-9a348b306a1d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:33.197009 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1231/2479] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:33.203847 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7bb37-fc37-42d5-bde1-c60456788112 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:33.221578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7bb37-fc37-42d5-bde1-c60456788112 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:33.228948 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ee68a-6978-485c-a927-14e6d0143d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:33.230503 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ee68a-6978-485c-a927-14e6d0143d00 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:33.230946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ee68a-6978-485c-a927-14e6d0143d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:33.231427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ee68a-6978-485c-a927-14e6d0143d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:33.232680 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ee68a-6978-485c-a927-14e6d0143d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:33.235476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ee68a-6978-485c-a927-14e6d0143d00 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:33.253464 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1249/2480] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:33.258090 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8acb4-5851-4b26-a50a-9a348b306a1d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:33.270246 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8acb4-5851-4b26-a50a-9a348b306a1d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:33.275385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c980aa-a7d2-41a6-a1fe-9b93ffd73c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:33.276942 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c980aa-a7d2-41a6-a1fe-9b93ffd73c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:33.277398 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c980aa-a7d2-41a6-a1fe-9b93ffd73c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:33.277687 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c980aa-a7d2-41a6-a1fe-9b93ffd73c81 admin admin] PATH_INFO: `/v3/projects/78493a5f4ebf4fbaac1536ba28e0ab93/users/ca7c881b01e8472eb578ad8615082bde/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:33.296790 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c980aa-a7d2-41a6-a1fe-9b93ffd73c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78493a5f4ebf4fbaac1536ba28e0ab93, user_id=ca7c881b01e8472eb578ad8615082bde, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:33.299580 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c980aa-a7d2-41a6-a1fe-9b93ffd73c81 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:33.317841 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1232/2481] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:33 2022] PUT /identity/v3/projects/78493a5f4ebf4fbaac1536ba28e0ab93/users/ca7c881b01e8472eb578ad8615082bde/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:33.336919 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13be62-db58-4dfa-a67d-062666bf4dee None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:33.337795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13be62-db58-4dfa-a67d-062666bf4dee None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:33.339080 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13be62-db58-4dfa-a67d-062666bf4dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:33.380157 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-cc13be62-db58-4dfa-a67d-062666bf4dee None None] MFA Rules not processed for user `ca7c881b01e8472eb578ad8615082bde`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:33.382509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-cc13be62-db58-4dfa-a67d-062666bf4dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:33.476395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1250/2482] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:33 2022] POST /identity/v3/auth/tokens => generated 3220 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:33.481563 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c980aa-a7d2-41a6-a1fe-9b93ffd73c81 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:33.498038 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:33.577404 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c980aa-a7d2-41a6-a1fe-9b93ffd73c81 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:33.585738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1e520-808a-45b8-9c7b-5fcbd5893515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:33.586890 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1e520-808a-45b8-9c7b-5fcbd5893515 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:33.587406 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1e520-808a-45b8-9c7b-5fcbd5893515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:33.587926 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1e520-808a-45b8-9c7b-5fcbd5893515 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:33.589912 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1e520-808a-45b8-9c7b-5fcbd5893515 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:33.593788 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1e520-808a-45b8-9c7b-5fcbd5893515 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:33.645629 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1233/2483] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:37:33 2022] POST /identity/v3/projects => generated 355 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:33.649981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13be62-db58-4dfa-a67d-062666bf4dee None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:33.664960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13be62-db58-4dfa-a67d-062666bf4dee None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:33.743482 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f773e-9723-4177-8aa5-dd219805818a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:33.744286 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f773e-9723-4177-8aa5-dd219805818a admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:33.744638 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f773e-9723-4177-8aa5-dd219805818a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:33.745458 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f773e-9723-4177-8aa5-dd219805818a admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:33.746466 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f773e-9723-4177-8aa5-dd219805818a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:33.748707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f773e-9723-4177-8aa5-dd219805818a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:33.785184 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1251/2484] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:37:33 2022] POST /identity/v3/users => generated 284 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:37:33.786963 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1e520-808a-45b8-9c7b-5fcbd5893515 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:33.801572 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1e520-808a-45b8-9c7b-5fcbd5893515 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:33.807281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a896db-2666-4396-a066-6cd6151249b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:33.808045 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a896db-2666-4396-a066-6cd6151249b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:33.808474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a896db-2666-4396-a066-6cd6151249b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:33.808783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a896db-2666-4396-a066-6cd6151249b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:33.813586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a896db-2666-4396-a066-6cd6151249b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:33.815254 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a896db-2666-4396-a066-6cd6151249b1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:33.827860 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1234/2485] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:37:33.832520 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f773e-9723-4177-8aa5-dd219805818a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:33.844775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f773e-9723-4177-8aa5-dd219805818a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:33.850207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b664f2-845a-449b-b4aa-161aca337659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:33.851117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b664f2-845a-449b-b4aa-161aca337659 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:33.851408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b664f2-845a-449b-b4aa-161aca337659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:33.851695 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b664f2-845a-449b-b4aa-161aca337659 admin admin] PATH_INFO: `/v3/projects/c101660b659148f2b7a522dd22d054be/users/b9ee2ce106334f6f8c06759eb8fe92e7/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:33.869877 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b664f2-845a-449b-b4aa-161aca337659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c101660b659148f2b7a522dd22d054be, user_id=b9ee2ce106334f6f8c06759eb8fe92e7, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:33.872629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b664f2-845a-449b-b4aa-161aca337659 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:33.889257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1252/2486] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:33 2022] PUT /identity/v3/projects/c101660b659148f2b7a522dd22d054be/users/b9ee2ce106334f6f8c06759eb8fe92e7/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:33.895395 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a896db-2666-4396-a066-6cd6151249b1 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:33.907453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a896db-2666-4396-a066-6cd6151249b1 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:33.985987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c9d6c-c9b8-4039-a421-7fddb3362078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:33.986636 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c9d6c-c9b8-4039-a421-7fddb3362078 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:33.986948 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c9d6c-c9b8-4039-a421-7fddb3362078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:33.987258 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c9d6c-c9b8-4039-a421-7fddb3362078 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:33.988203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c9d6c-c9b8-4039-a421-7fddb3362078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:33.989897 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c9d6c-c9b8-4039-a421-7fddb3362078 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:34.003287 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1235/2487] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:34.009959 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b664f2-845a-449b-b4aa-161aca337659 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:34.030655 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b664f2-845a-449b-b4aa-161aca337659 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:34.036988 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c49be5-f158-4f3d-badd-9feae220ac2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:34.037938 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c49be5-f158-4f3d-badd-9feae220ac2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:34.038229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c49be5-f158-4f3d-badd-9feae220ac2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:34.038514 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c49be5-f158-4f3d-badd-9feae220ac2e admin admin] PATH_INFO: `/v3/projects/c101660b659148f2b7a522dd22d054be/users/b9ee2ce106334f6f8c06759eb8fe92e7/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:34.040613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c49be5-f158-4f3d-badd-9feae220ac2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c101660b659148f2b7a522dd22d054be, user_id=b9ee2ce106334f6f8c06759eb8fe92e7, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:34.043827 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c49be5-f158-4f3d-badd-9feae220ac2e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:34.056039 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1253/2488] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:34 2022] PUT /identity/v3/projects/c101660b659148f2b7a522dd22d054be/users/b9ee2ce106334f6f8c06759eb8fe92e7/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:34.077518 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ee873-48cb-4fc9-aefc-48dc45263cd0 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:34.078031 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ee873-48cb-4fc9-aefc-48dc45263cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:34.078543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ee873-48cb-4fc9-aefc-48dc45263cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:34.128278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-5d3ee873-48cb-4fc9-aefc-48dc45263cd0 None None] MFA Rules not processed for user `b9ee2ce106334f6f8c06759eb8fe92e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:34.130094 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5d3ee873-48cb-4fc9-aefc-48dc45263cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:34.237629 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1236/2489] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:34 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:37:34.251494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c49be5-f158-4f3d-badd-9feae220ac2e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:34.254278 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c9c49be5-f158-4f3d-badd-9feae220ac2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:34.271521 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:34.366572 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c49be5-f158-4f3d-badd-9feae220ac2e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:34.372448 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28534a84-f241-4bad-a9c2-870a057bedee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:34.373186 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28534a84-f241-4bad-a9c2-870a057bedee service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:34.373555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28534a84-f241-4bad-a9c2-870a057bedee service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:34.373880 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28534a84-f241-4bad-a9c2-870a057bedee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:34.384826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28534a84-f241-4bad-a9c2-870a057bedee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:34.387261 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28534a84-f241-4bad-a9c2-870a057bedee service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:34.398125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1254/2490] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:34 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:37:34.488932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ee873-48cb-4fc9-aefc-48dc45263cd0 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:34.503217 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ee873-48cb-4fc9-aefc-48dc45263cd0 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:34.508035 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427acb67-0120-4950-9611-6305ce67aec0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:34.508770 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427acb67-0120-4950-9611-6305ce67aec0 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:34.509116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427acb67-0120-4950-9611-6305ce67aec0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:34.509553 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427acb67-0120-4950-9611-6305ce67aec0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:34.519721 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:34.583031 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427acb67-0120-4950-9611-6305ce67aec0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:34.584671 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427acb67-0120-4950-9611-6305ce67aec0 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:34.595655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1237/2491] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:34 2022] GET /identity/v3/auth/tokens => generated 3220 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:37:39.494684 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28534a84-f241-4bad-a9c2-870a057bedee service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:39.495783 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-28534a84-f241-4bad-a9c2-870a057bedee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:39.569101 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28534a84-f241-4bad-a9c2-870a057bedee service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:39.574254 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7a65e-9227-47a0-9b95-60abec5f69ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:39.575091 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7a65e-9227-47a0-9b95-60abec5f69ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:39.575384 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7a65e-9227-47a0-9b95-60abec5f69ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:39.575657 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7a65e-9227-47a0-9b95-60abec5f69ff admin admin] PATH_INFO: `/v3/users/f99a5aaef1a446ec819ba675dd5d47dc` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:39.577001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7a65e-9227-47a0-9b95-60abec5f69ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=f99a5aaef1a446ec819ba675dd5d47dc)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:39.579004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7a65e-9227-47a0-9b95-60abec5f69ff admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:39.632252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4ff7a65e-9227-47a0-9b95-60abec5f69ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f99a5aaef1a446ec819ba675dd5d47dc'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:39.632962 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4ff7a65e-9227-47a0-9b95-60abec5f69ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f99a5aaef1a446ec819ba675dd5d47dc'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:39.647910 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4ff7a65e-9227-47a0-9b95-60abec5f69ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f99a5aaef1a446ec819ba675dd5d47dc'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:39.648666 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4ff7a65e-9227-47a0-9b95-60abec5f69ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f99a5aaef1a446ec819ba675dd5d47dc'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:39.658200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4ff7a65e-9227-47a0-9b95-60abec5f69ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f99a5aaef1a446ec819ba675dd5d47dc'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:39.674033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1255/2492] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:37:39 2022] DELETE /identity/v3/users/f99a5aaef1a446ec819ba675dd5d47dc => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:39.678968 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f52d6d-a362-4e3f-873e-db6d71c41361 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:39.679615 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f52d6d-a362-4e3f-873e-db6d71c41361 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:39.679615 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f52d6d-a362-4e3f-873e-db6d71c41361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:39.698212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-59f52d6d-a362-4e3f-873e-db6d71c41361 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:39.699581 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-59f52d6d-a362-4e3f-873e-db6d71c41361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:39.770355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1238/2493] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:39.782483 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7a65e-9227-47a0-9b95-60abec5f69ff admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:39.783994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4ff7a65e-9227-47a0-9b95-60abec5f69ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:39.823608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:39.854810 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7a65e-9227-47a0-9b95-60abec5f69ff admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:39.859263 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc9d26-8937-4aa4-aa6f-3a0f057a9315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:39.859975 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc9d26-8937-4aa4-aa6f-3a0f057a9315 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:39.860262 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc9d26-8937-4aa4-aa6f-3a0f057a9315 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:39.860543 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc9d26-8937-4aa4-aa6f-3a0f057a9315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:39.871283 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc9d26-8937-4aa4-aa6f-3a0f057a9315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:39.872751 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc9d26-8937-4aa4-aa6f-3a0f057a9315 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:39.883674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1256/2494] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:37:40.294772 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f52d6d-a362-4e3f-873e-db6d71c41361 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:40.297741 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-59f52d6d-a362-4e3f-873e-db6d71c41361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:40.324765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f52d6d-a362-4e3f-873e-db6d71c41361 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:40.334071 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf47ad-d047-43f3-9e68-48c8c7c2c2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:40.335658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf47ad-d047-43f3-9e68-48c8c7c2c2e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:40.336238 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf47ad-d047-43f3-9e68-48c8c7c2c2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:40.336819 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf47ad-d047-43f3-9e68-48c8c7c2c2e3 admin admin] PATH_INFO: `/v3/projects/4a0a5b597c3743b1ac55fd40bd141ac7` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:40.340098 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf47ad-d047-43f3-9e68-48c8c7c2c2e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a0a5b597c3743b1ac55fd40bd141ac7)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:40.345268 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf47ad-d047-43f3-9e68-48c8c7c2c2e3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:40.423036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-cabf47ad-d047-43f3-9e68-48c8c7c2c2e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a0a5b597c3743b1ac55fd40bd141ac7', 'request_id': 'req-cabf47ad-d047-43f3-9e68-48c8c7c2c2e3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:40.433649 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-cabf47ad-d047-43f3-9e68-48c8c7c2c2e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a0a5b597c3743b1ac55fd40bd141ac7', 'request_id': 'req-cabf47ad-d047-43f3-9e68-48c8c7c2c2e3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:40.443928 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-cabf47ad-d047-43f3-9e68-48c8c7c2c2e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a0a5b597c3743b1ac55fd40bd141ac7', 'request_id': 'req-cabf47ad-d047-43f3-9e68-48c8c7c2c2e3'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:40.465521 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-cabf47ad-d047-43f3-9e68-48c8c7c2c2e3 admin admin] The token cache is being invalidate because project 4a0a5b597c3743b1ac55fd40bd141ac7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:37:40.466065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-cabf47ad-d047-43f3-9e68-48c8c7c2c2e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:40.467210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-cabf47ad-d047-43f3-9e68-48c8c7c2c2e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:40.470111 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1239/2495] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:37:40 2022] DELETE /identity/v3/projects/4a0a5b597c3743b1ac55fd40bd141ac7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:40.476782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25c288-564f-4108-b1da-d62375a59dc0 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:40.477219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25c288-564f-4108-b1da-d62375a59dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:40.477720 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25c288-564f-4108-b1da-d62375a59dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:40.506553 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8a25c288-564f-4108-b1da-d62375a59dc0 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:40.508686 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8a25c288-564f-4108-b1da-d62375a59dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:40.613003 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1257/2496] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:40.622566 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ea531-7910-4b51-8ba2-65dd432cdf3e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:40.622988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ea531-7910-4b51-8ba2-65dd432cdf3e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:40.623482 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ea531-7910-4b51-8ba2-65dd432cdf3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:40.647877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3f2ea531-7910-4b51-8ba2-65dd432cdf3e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:40.649125 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3f2ea531-7910-4b51-8ba2-65dd432cdf3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:40.659051 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1240/2497] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:37:40.664306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25c288-564f-4108-b1da-d62375a59dc0 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:40.676342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25c288-564f-4108-b1da-d62375a59dc0 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:40.681738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f13f2-0bab-404e-b2b9-76f25cee7251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:40.683144 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f13f2-0bab-404e-b2b9-76f25cee7251 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:40.683457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f13f2-0bab-404e-b2b9-76f25cee7251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:40.683739 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f13f2-0bab-404e-b2b9-76f25cee7251 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:40.684513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f13f2-0bab-404e-b2b9-76f25cee7251 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:37:40.684825 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f13f2-0bab-404e-b2b9-76f25cee7251 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:40.686521 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f13f2-0bab-404e-b2b9-76f25cee7251 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:40.701506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1258/2498] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:37:40.706431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ea531-7910-4b51-8ba2-65dd432cdf3e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:40.718640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ea531-7910-4b51-8ba2-65dd432cdf3e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:40.724769 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2024b7c-af28-4f7e-b7cd-7118486df892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:40.725711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2024b7c-af28-4f7e-b7cd-7118486df892 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:40.726103 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2024b7c-af28-4f7e-b7cd-7118486df892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:40.726593 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2024b7c-af28-4f7e-b7cd-7118486df892 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:40.727467 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2024b7c-af28-4f7e-b7cd-7118486df892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:40.730002 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2024b7c-af28-4f7e-b7cd-7118486df892 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:40.763455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1241/2499] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:37:40 2022] POST /identity/v3/projects => generated 403 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:40.767897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f13f2-0bab-404e-b2b9-76f25cee7251 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:40.779979 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f13f2-0bab-404e-b2b9-76f25cee7251 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:40.849800 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b27bd-66d5-4991-8683-9c7b815b2b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:40.850565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b27bd-66d5-4991-8683-9c7b815b2b63 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:40.850897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b27bd-66d5-4991-8683-9c7b815b2b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:40.851219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b27bd-66d5-4991-8683-9c7b815b2b63 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:40.852011 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b27bd-66d5-4991-8683-9c7b815b2b63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:40.854260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b27bd-66d5-4991-8683-9c7b815b2b63 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:40.879002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1259/2500] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:37:40 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:37:40.884086 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2024b7c-af28-4f7e-b7cd-7118486df892 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:40.890506 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:40.898893 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2024b7c-af28-4f7e-b7cd-7118486df892 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:40.906802 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b3926-ba12-4dd9-90b1-7db80b9d48fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:40.907528 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b3926-ba12-4dd9-90b1-7db80b9d48fc admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:40.907819 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b3926-ba12-4dd9-90b1-7db80b9d48fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:40.908103 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b3926-ba12-4dd9-90b1-7db80b9d48fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:40.908852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b3926-ba12-4dd9-90b1-7db80b9d48fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:40.910560 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b3926-ba12-4dd9-90b1-7db80b9d48fc admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:40.923672 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1242/2501] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:40.928677 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b27bd-66d5-4991-8683-9c7b815b2b63 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:40.947313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b27bd-66d5-4991-8683-9c7b815b2b63 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:40.955940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85adfa-1697-4d64-9ce4-1129e5765dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:40.957079 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85adfa-1697-4d64-9ce4-1129e5765dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:40.957644 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85adfa-1697-4d64-9ce4-1129e5765dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:40.958147 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85adfa-1697-4d64-9ce4-1129e5765dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:40.959455 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85adfa-1697-4d64-9ce4-1129e5765dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:40.962373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85adfa-1697-4d64-9ce4-1129e5765dd6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:40.981192 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1260/2502] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:40.986877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b3926-ba12-4dd9-90b1-7db80b9d48fc admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:40.999318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b3926-ba12-4dd9-90b1-7db80b9d48fc admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:41.004532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5067a-b525-4579-8226-304b4080e797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:41.005475 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5067a-b525-4579-8226-304b4080e797 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:41.005763 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5067a-b525-4579-8226-304b4080e797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:41.006047 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5067a-b525-4579-8226-304b4080e797 admin admin] PATH_INFO: `/v3/projects/6fd892437ab94cfdb71e92b23fac9744/users/574a0e5e6d874c15bc93ecfe1a9cbff4/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:41.027518 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e5067a-b525-4579-8226-304b4080e797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fd892437ab94cfdb71e92b23fac9744, user_id=574a0e5e6d874c15bc93ecfe1a9cbff4, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:41.033391 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e5067a-b525-4579-8226-304b4080e797 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:41.052844 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1243/2503] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:40 2022] PUT /identity/v3/projects/6fd892437ab94cfdb71e92b23fac9744/users/574a0e5e6d874c15bc93ecfe1a9cbff4/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:41.059729 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864d833-1500-4da7-9c8d-1b89c9c423af None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:41.060132 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864d833-1500-4da7-9c8d-1b89c9c423af None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:41.060559 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864d833-1500-4da7-9c8d-1b89c9c423af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:41.085028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-0864d833-1500-4da7-9c8d-1b89c9c423af None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:41.086502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0864d833-1500-4da7-9c8d-1b89c9c423af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:41.132366 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:41.172027 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1261/2504] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:41.184460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5067a-b525-4579-8226-304b4080e797 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:41.186465 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a0e5067a-b525-4579-8226-304b4080e797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:41.267801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5067a-b525-4579-8226-304b4080e797 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:41.274079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1bcf0-c838-49ca-90c6-476e1a940c01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:41.275183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1bcf0-c838-49ca-90c6-476e1a940c01 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:41.275672 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1bcf0-c838-49ca-90c6-476e1a940c01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:41.276150 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1bcf0-c838-49ca-90c6-476e1a940c01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:41.293219 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1bcf0-c838-49ca-90c6-476e1a940c01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:41.295046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1bcf0-c838-49ca-90c6-476e1a940c01 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:41.315153 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1244/2505] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:37:42.263460 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084fd73-35fe-44a4-8b5c-dc4ac07a0d05 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:42.264323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084fd73-35fe-44a4-8b5c-dc4ac07a0d05 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:42.264323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084fd73-35fe-44a4-8b5c-dc4ac07a0d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:42.313867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a084fd73-35fe-44a4-8b5c-dc4ac07a0d05 None None] MFA Rules not processed for user `574a0e5e6d874c15bc93ecfe1a9cbff4`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:42.315187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a084fd73-35fe-44a4-8b5c-dc4ac07a0d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:42.480158 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1262/2506] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:37:42.484587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1bcf0-c838-49ca-90c6-476e1a940c01 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:42.498827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1bcf0-c838-49ca-90c6-476e1a940c01 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:42.504658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d865c2-226c-4f46-ab2f-d9bba13e12c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:42.505383 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d865c2-226c-4f46-ab2f-d9bba13e12c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:42.505723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d865c2-226c-4f46-ab2f-d9bba13e12c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:42.506172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d865c2-226c-4f46-ab2f-d9bba13e12c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:42.506901 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d865c2-226c-4f46-ab2f-d9bba13e12c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:42.509275 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d865c2-226c-4f46-ab2f-d9bba13e12c3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:42.545193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1245/2507] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:37:42 2022] POST /identity/v3/projects => generated 405 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:42.549571 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084fd73-35fe-44a4-8b5c-dc4ac07a0d05 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:42.563681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084fd73-35fe-44a4-8b5c-dc4ac07a0d05 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:42.668841 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba847ac3-46fe-4321-9d62-f9ccd162cf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:42.670133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba847ac3-46fe-4321-9d62-f9ccd162cf7f admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:42.670636 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba847ac3-46fe-4321-9d62-f9ccd162cf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:42.671141 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba847ac3-46fe-4321-9d62-f9ccd162cf7f admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:42.672465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba847ac3-46fe-4321-9d62-f9ccd162cf7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:42.676338 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba847ac3-46fe-4321-9d62-f9ccd162cf7f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:42.710793 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1263/2508] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:37:42 2022] POST /identity/v3/users => generated 309 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:42.715767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d865c2-226c-4f46-ab2f-d9bba13e12c3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:42.735893 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d865c2-226c-4f46-ab2f-d9bba13e12c3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:42.744212 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246f8f0-aea0-4c04-afc3-eba37f640a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:42.745536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246f8f0-aea0-4c04-afc3-eba37f640a61 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:42.745993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246f8f0-aea0-4c04-afc3-eba37f640a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:42.746494 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246f8f0-aea0-4c04-afc3-eba37f640a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:42.747917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246f8f0-aea0-4c04-afc3-eba37f640a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:42.751170 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246f8f0-aea0-4c04-afc3-eba37f640a61 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:42.773151 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1246/2509] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:42.778929 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba847ac3-46fe-4321-9d62-f9ccd162cf7f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:42.792482 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba847ac3-46fe-4321-9d62-f9ccd162cf7f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:42.800466 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318e925-358d-440e-86ba-856fcac4682c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:42.801399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318e925-358d-440e-86ba-856fcac4682c admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:42.801678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318e925-358d-440e-86ba-856fcac4682c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:42.801951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318e925-358d-440e-86ba-856fcac4682c admin admin] PATH_INFO: `/v3/projects/757756bde8c241c9917cdcf277b306ea/users/f5b3f01f46564ed180a8ed90968b9127/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:42.819744 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6318e925-358d-440e-86ba-856fcac4682c admin admin] RBAC: Authorizing `identity:create_grant(project_id=757756bde8c241c9917cdcf277b306ea, user_id=f5b3f01f46564ed180a8ed90968b9127, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:42.822485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6318e925-358d-440e-86ba-856fcac4682c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:42.839662 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1264/2510] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:42 2022] PUT /identity/v3/projects/757756bde8c241c9917cdcf277b306ea/users/f5b3f01f46564ed180a8ed90968b9127/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:42.845069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246f8f0-aea0-4c04-afc3-eba37f640a61 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:42.857755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246f8f0-aea0-4c04-afc3-eba37f640a61 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:42.930282 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6132b-210c-4703-831e-9b22558c52ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:42.931057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6132b-210c-4703-831e-9b22558c52ae admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:42.931538 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6132b-210c-4703-831e-9b22558c52ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:42.931813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6132b-210c-4703-831e-9b22558c52ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:42.933162 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6132b-210c-4703-831e-9b22558c52ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:42.935175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6132b-210c-4703-831e-9b22558c52ae admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:42.946651 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1247/2511] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:42.952280 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318e925-358d-440e-86ba-856fcac4682c admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:42.964501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318e925-358d-440e-86ba-856fcac4682c admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:42.969393 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42664d-af8d-48ac-b4a8-5dcd6a20dbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:42.970332 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42664d-af8d-48ac-b4a8-5dcd6a20dbde admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:42.970615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42664d-af8d-48ac-b4a8-5dcd6a20dbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:42.970896 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42664d-af8d-48ac-b4a8-5dcd6a20dbde admin admin] PATH_INFO: `/v3/projects/757756bde8c241c9917cdcf277b306ea/users/f5b3f01f46564ed180a8ed90968b9127/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:42.973004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42664d-af8d-48ac-b4a8-5dcd6a20dbde admin admin] RBAC: Authorizing `identity:create_grant(project_id=757756bde8c241c9917cdcf277b306ea, user_id=f5b3f01f46564ed180a8ed90968b9127, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:42.975780 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42664d-af8d-48ac-b4a8-5dcd6a20dbde admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:42.988014 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1265/2512] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:42 2022] PUT /identity/v3/projects/757756bde8c241c9917cdcf277b306ea/users/f5b3f01f46564ed180a8ed90968b9127/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:44.001624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eade364-8ff4-4cd9-9ad4-0165d4f83eec None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:44.002501 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eade364-8ff4-4cd9-9ad4-0165d4f83eec None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:44.002666 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eade364-8ff4-4cd9-9ad4-0165d4f83eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:44.045315 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3eade364-8ff4-4cd9-9ad4-0165d4f83eec None None] MFA Rules not processed for user `f5b3f01f46564ed180a8ed90968b9127`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:44.047978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3eade364-8ff4-4cd9-9ad4-0165d4f83eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:44.094456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:44.201418 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1248/2513] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:43 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:37:45.815223 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42664d-af8d-48ac-b4a8-5dcd6a20dbde admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:45.866556 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:45.935486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42664d-af8d-48ac-b4a8-5dcd6a20dbde admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:45.941417 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd92a45-9fe2-4af2-b9e3-b40ad6197d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:45.942463 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd92a45-9fe2-4af2-b9e3-b40ad6197d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:45.944228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd92a45-9fe2-4af2-b9e3-b40ad6197d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:45.944228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd92a45-9fe2-4af2-b9e3-b40ad6197d7a admin admin] PATH_INFO: `/v3/users/574a0e5e6d874c15bc93ecfe1a9cbff4` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:45.945203 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd92a45-9fe2-4af2-b9e3-b40ad6197d7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=574a0e5e6d874c15bc93ecfe1a9cbff4)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:45.948681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd92a45-9fe2-4af2-b9e3-b40ad6197d7a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:46.021689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9dd92a45-9fe2-4af2-b9e3-b40ad6197d7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '574a0e5e6d874c15bc93ecfe1a9cbff4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:46.022653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9dd92a45-9fe2-4af2-b9e3-b40ad6197d7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '574a0e5e6d874c15bc93ecfe1a9cbff4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:46.055102 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9dd92a45-9fe2-4af2-b9e3-b40ad6197d7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '574a0e5e6d874c15bc93ecfe1a9cbff4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:46.056036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9dd92a45-9fe2-4af2-b9e3-b40ad6197d7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '574a0e5e6d874c15bc93ecfe1a9cbff4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:46.074247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9dd92a45-9fe2-4af2-b9e3-b40ad6197d7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '574a0e5e6d874c15bc93ecfe1a9cbff4'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:46.090921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1266/2514] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:37:45 2022] DELETE /identity/v3/users/574a0e5e6d874c15bc93ecfe1a9cbff4 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:46.308488 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eade364-8ff4-4cd9-9ad4-0165d4f83eec None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:46.327886 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eade364-8ff4-4cd9-9ad4-0165d4f83eec None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:46.397623 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d85fb-969f-4693-aacd-a0dce35a1cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:46.398616 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d85fb-969f-4693-aacd-a0dce35a1cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:46.398935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d85fb-969f-4693-aacd-a0dce35a1cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:46.399257 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d85fb-969f-4693-aacd-a0dce35a1cac admin admin] PATH_INFO: `/v3/projects/6fd892437ab94cfdb71e92b23fac9744` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:46.400657 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d85fb-969f-4693-aacd-a0dce35a1cac admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fd892437ab94cfdb71e92b23fac9744)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:46.404027 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d85fb-969f-4693-aacd-a0dce35a1cac admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:46.474553 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6c2d85fb-969f-4693-aacd-a0dce35a1cac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fd892437ab94cfdb71e92b23fac9744', 'request_id': 'req-6c2d85fb-969f-4693-aacd-a0dce35a1cac'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:46.484937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6c2d85fb-969f-4693-aacd-a0dce35a1cac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fd892437ab94cfdb71e92b23fac9744', 'request_id': 'req-6c2d85fb-969f-4693-aacd-a0dce35a1cac'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:46.491943 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6c2d85fb-969f-4693-aacd-a0dce35a1cac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fd892437ab94cfdb71e92b23fac9744', 'request_id': 'req-6c2d85fb-969f-4693-aacd-a0dce35a1cac'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:46.512474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6c2d85fb-969f-4693-aacd-a0dce35a1cac admin admin] The token cache is being invalidate because project 6fd892437ab94cfdb71e92b23fac9744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:37:46.513612 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6c2d85fb-969f-4693-aacd-a0dce35a1cac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:46.513786 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6c2d85fb-969f-4693-aacd-a0dce35a1cac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:46.515527 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1249/2515] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:37:46 2022] DELETE /identity/v3/projects/6fd892437ab94cfdb71e92b23fac9744 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:46.519821 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd92a45-9fe2-4af2-b9e3-b40ad6197d7a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:46.521589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9dd92a45-9fe2-4af2-b9e3-b40ad6197d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:46.538091 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd92a45-9fe2-4af2-b9e3-b40ad6197d7a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:46.617002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529778aa-6769-47a6-a8ee-9193654614ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:46.618140 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529778aa-6769-47a6-a8ee-9193654614ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:46.618468 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529778aa-6769-47a6-a8ee-9193654614ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:46.618806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529778aa-6769-47a6-a8ee-9193654614ee admin admin] PATH_INFO: `/v3/users/f5b3f01f46564ed180a8ed90968b9127` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:46.620287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529778aa-6769-47a6-a8ee-9193654614ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5b3f01f46564ed180a8ed90968b9127)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:46.622428 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529778aa-6769-47a6-a8ee-9193654614ee admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:46.673494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:46.676512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-529778aa-6769-47a6-a8ee-9193654614ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5b3f01f46564ed180a8ed90968b9127'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:46.678035 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-529778aa-6769-47a6-a8ee-9193654614ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5b3f01f46564ed180a8ed90968b9127'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:46.694165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-529778aa-6769-47a6-a8ee-9193654614ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5b3f01f46564ed180a8ed90968b9127'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:46.694939 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-529778aa-6769-47a6-a8ee-9193654614ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5b3f01f46564ed180a8ed90968b9127'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:46.705695 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-529778aa-6769-47a6-a8ee-9193654614ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5b3f01f46564ed180a8ed90968b9127'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:46.721070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1267/2516] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:37:46 2022] DELETE /identity/v3/users/f5b3f01f46564ed180a8ed90968b9127 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:46.911509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d85fb-969f-4693-aacd-a0dce35a1cac admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:46.913423 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6c2d85fb-969f-4693-aacd-a0dce35a1cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:46.932925 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d85fb-969f-4693-aacd-a0dce35a1cac admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:46.989683 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:47.006298 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a69c6c-7c9d-4a7a-af3c-473203bdf6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:47.007178 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a69c6c-7c9d-4a7a-af3c-473203bdf6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:47.007520 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a69c6c-7c9d-4a7a-af3c-473203bdf6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:47.007804 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a69c6c-7c9d-4a7a-af3c-473203bdf6ee admin admin] PATH_INFO: `/v3/projects/757756bde8c241c9917cdcf277b306ea` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:47.009029 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a69c6c-7c9d-4a7a-af3c-473203bdf6ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=757756bde8c241c9917cdcf277b306ea)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:47.011800 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a69c6c-7c9d-4a7a-af3c-473203bdf6ee admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:47.070369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b7a69c6c-7c9d-4a7a-af3c-473203bdf6ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '757756bde8c241c9917cdcf277b306ea', 'request_id': 'req-b7a69c6c-7c9d-4a7a-af3c-473203bdf6ee'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:47.082166 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b7a69c6c-7c9d-4a7a-af3c-473203bdf6ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '757756bde8c241c9917cdcf277b306ea', 'request_id': 'req-b7a69c6c-7c9d-4a7a-af3c-473203bdf6ee'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:47.094291 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b7a69c6c-7c9d-4a7a-af3c-473203bdf6ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '757756bde8c241c9917cdcf277b306ea', 'request_id': 'req-b7a69c6c-7c9d-4a7a-af3c-473203bdf6ee'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:47.110476 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b7a69c6c-7c9d-4a7a-af3c-473203bdf6ee admin admin] The token cache is being invalidate because project 757756bde8c241c9917cdcf277b306ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:37:47.110780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b7a69c6c-7c9d-4a7a-af3c-473203bdf6ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:47.112390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-b7a69c6c-7c9d-4a7a-af3c-473203bdf6ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:47.113114 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1250/2517] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:37:46 2022] DELETE /identity/v3/projects/757756bde8c241c9917cdcf277b306ea => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:47.118497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894223c6-ef22-4891-8072-5252bff04776 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:47.119142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894223c6-ef22-4891-8072-5252bff04776 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:47.119142 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894223c6-ef22-4891-8072-5252bff04776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:47.140007 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-894223c6-ef22-4891-8072-5252bff04776 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:47.141313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-894223c6-ef22-4891-8072-5252bff04776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:47.247957 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1268/2518] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:37:47.250144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882030d-adfc-464b-9f11-3c7cced7a0f7 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:47.250444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882030d-adfc-464b-9f11-3c7cced7a0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:47.250813 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882030d-adfc-464b-9f11-3c7cced7a0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:47.271389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-2882030d-adfc-464b-9f11-3c7cced7a0f7 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:47.272687 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2882030d-adfc-464b-9f11-3c7cced7a0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:47.282201 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1251/2519] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:37:47.291210 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894223c6-ef22-4891-8072-5252bff04776 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:47.303097 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894223c6-ef22-4891-8072-5252bff04776 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:47.308256 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81e6d9-85c7-42db-bfea-afa2ba101bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:47.309007 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81e6d9-85c7-42db-bfea-afa2ba101bee admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:47.309385 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81e6d9-85c7-42db-bfea-afa2ba101bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:47.309718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81e6d9-85c7-42db-bfea-afa2ba101bee admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:47.310519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81e6d9-85c7-42db-bfea-afa2ba101bee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:37:47.310885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81e6d9-85c7-42db-bfea-afa2ba101bee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:47.312574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81e6d9-85c7-42db-bfea-afa2ba101bee admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:47.327263 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1269/2520] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:37:47.332218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882030d-adfc-464b-9f11-3c7cced7a0f7 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:47.342705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882030d-adfc-464b-9f11-3c7cced7a0f7 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:47.347770 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63e837-a7c8-4924-8f71-110f9a422a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:47.348436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63e837-a7c8-4924-8f71-110f9a422a67 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:47.348721 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63e837-a7c8-4924-8f71-110f9a422a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:47.348994 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63e837-a7c8-4924-8f71-110f9a422a67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:47.349837 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63e837-a7c8-4924-8f71-110f9a422a67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:47.351860 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63e837-a7c8-4924-8f71-110f9a422a67 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:47.383290 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1252/2521] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:37:47 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:47.390162 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81e6d9-85c7-42db-bfea-afa2ba101bee admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:47.402000 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81e6d9-85c7-42db-bfea-afa2ba101bee admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:47.465166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264f78b-328c-4960-81fa-a93e252d5768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:47.465887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264f78b-328c-4960-81fa-a93e252d5768 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:47.466176 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264f78b-328c-4960-81fa-a93e252d5768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:47.466458 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264f78b-328c-4960-81fa-a93e252d5768 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:47.467178 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264f78b-328c-4960-81fa-a93e252d5768 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:47.469134 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264f78b-328c-4960-81fa-a93e252d5768 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:47.495049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1270/2522] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:37:47 2022] POST /identity/v3/users => generated 282 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:47.500215 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63e837-a7c8-4924-8f71-110f9a422a67 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:47.512395 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63e837-a7c8-4924-8f71-110f9a422a67 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:47.517610 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bf698-aa30-4a71-a328-99305124bc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:47.518295 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bf698-aa30-4a71-a328-99305124bc11 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:47.518584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bf698-aa30-4a71-a328-99305124bc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:47.518866 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bf698-aa30-4a71-a328-99305124bc11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:47.519604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bf698-aa30-4a71-a328-99305124bc11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:47.521271 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bf698-aa30-4a71-a328-99305124bc11 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:47.533075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1253/2523] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:37:47.537146 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264f78b-328c-4960-81fa-a93e252d5768 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:47.548149 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264f78b-328c-4960-81fa-a93e252d5768 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:47.553399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9404c93-4031-4819-8c1e-38671320d10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:47.554133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9404c93-4031-4819-8c1e-38671320d10a admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:47.554460 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9404c93-4031-4819-8c1e-38671320d10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:47.554781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9404c93-4031-4819-8c1e-38671320d10a admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:47.555654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9404c93-4031-4819-8c1e-38671320d10a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:47.557847 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9404c93-4031-4819-8c1e-38671320d10a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:47.569961 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1271/2524] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:47.575805 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bf698-aa30-4a71-a328-99305124bc11 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:47.593551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bf698-aa30-4a71-a328-99305124bc11 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:47.601420 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccfe9c-e83c-4a14-b378-c561a1de51fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:47.602978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccfe9c-e83c-4a14-b378-c561a1de51fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:47.603477 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccfe9c-e83c-4a14-b378-c561a1de51fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:47.604533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccfe9c-e83c-4a14-b378-c561a1de51fa admin admin] PATH_INFO: `/v3/projects/25394d32b06a4a1a9accb081ac7b3f33/users/01285e1ba2514e4582c4979753efd215/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:47.631762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccfe9c-e83c-4a14-b378-c561a1de51fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=25394d32b06a4a1a9accb081ac7b3f33, user_id=01285e1ba2514e4582c4979753efd215, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:47.634675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccfe9c-e83c-4a14-b378-c561a1de51fa admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:47.652123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1254/2525] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:47 2022] PUT /identity/v3/projects/25394d32b06a4a1a9accb081ac7b3f33/users/01285e1ba2514e4582c4979753efd215/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:47.663948 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd65b2f-ae36-4eeb-9cb2-3faefa92f9a1 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:47.664460 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd65b2f-ae36-4eeb-9cb2-3faefa92f9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:47.664798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd65b2f-ae36-4eeb-9cb2-3faefa92f9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:47.702578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ffd65b2f-ae36-4eeb-9cb2-3faefa92f9a1 None None] MFA Rules not processed for user `01285e1ba2514e4582c4979753efd215`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:47.703824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ffd65b2f-ae36-4eeb-9cb2-3faefa92f9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:47.720133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:47.782981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1272/2526] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:47 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:37:47.787656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccfe9c-e83c-4a14-b378-c561a1de51fa admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:47.876685 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccfe9c-e83c-4a14-b378-c561a1de51fa admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:47.884049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e66ac2-799e-46ca-9cbf-facfda2b1c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:47.884806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e66ac2-799e-46ca-9cbf-facfda2b1c7a admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:47.886015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e66ac2-799e-46ca-9cbf-facfda2b1c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:47.886316 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e66ac2-799e-46ca-9cbf-facfda2b1c7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:47.887224 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e66ac2-799e-46ca-9cbf-facfda2b1c7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:47.889415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e66ac2-799e-46ca-9cbf-facfda2b1c7a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:47.912615 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:47.922281 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1255/2527] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:37:47 2022] POST /identity/v3/projects => generated 365 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:47.926021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd65b2f-ae36-4eeb-9cb2-3faefa92f9a1 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:47.943413 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd65b2f-ae36-4eeb-9cb2-3faefa92f9a1 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:48.012834 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cd645-eecb-4dd8-a64f-b88b12b530ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:48.013709 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cd645-eecb-4dd8-a64f-b88b12b530ea admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:48.013990 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cd645-eecb-4dd8-a64f-b88b12b530ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:48.014266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cd645-eecb-4dd8-a64f-b88b12b530ea admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:48.014974 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715cd645-eecb-4dd8-a64f-b88b12b530ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:48.016983 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715cd645-eecb-4dd8-a64f-b88b12b530ea admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:48.039821 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1273/2528] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:37:47 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:48.044302 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e66ac2-799e-46ca-9cbf-facfda2b1c7a admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:48.055244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e66ac2-799e-46ca-9cbf-facfda2b1c7a admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:48.060470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04d44e-c1ae-4bf4-9b40-6aba3cbd2c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:48.061184 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04d44e-c1ae-4bf4-9b40-6aba3cbd2c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:48.061502 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04d44e-c1ae-4bf4-9b40-6aba3cbd2c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:48.061941 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04d44e-c1ae-4bf4-9b40-6aba3cbd2c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:48.062685 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04d44e-c1ae-4bf4-9b40-6aba3cbd2c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:48.064418 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04d44e-c1ae-4bf4-9b40-6aba3cbd2c3b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:48.081157 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1256/2529] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:48.085586 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cd645-eecb-4dd8-a64f-b88b12b530ea admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:48.097525 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cd645-eecb-4dd8-a64f-b88b12b530ea admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:48.102505 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8b767-8b01-47e5-a793-ee548399d548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:48.103421 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8b767-8b01-47e5-a793-ee548399d548 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:48.103735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8b767-8b01-47e5-a793-ee548399d548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:48.104169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8b767-8b01-47e5-a793-ee548399d548 admin admin] PATH_INFO: `/v3/projects/ec8d1f2182464a5383ef3dbac2957378/users/0e66b00cb45b46e69bb54eaafeaf85d8/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:48.120346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8b767-8b01-47e5-a793-ee548399d548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8d1f2182464a5383ef3dbac2957378, user_id=0e66b00cb45b46e69bb54eaafeaf85d8, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:48.123082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8b767-8b01-47e5-a793-ee548399d548 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:48.137637 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1274/2530] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:48 2022] PUT /identity/v3/projects/ec8d1f2182464a5383ef3dbac2957378/users/0e66b00cb45b46e69bb54eaafeaf85d8/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:48.142596 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04d44e-c1ae-4bf4-9b40-6aba3cbd2c3b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:48.155032 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04d44e-c1ae-4bf4-9b40-6aba3cbd2c3b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:48.218141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e238bcf-afd3-4052-aaf0-8f91dd4bf56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:48.218825 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e238bcf-afd3-4052-aaf0-8f91dd4bf56b admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:48.219116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e238bcf-afd3-4052-aaf0-8f91dd4bf56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:48.219404 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e238bcf-afd3-4052-aaf0-8f91dd4bf56b admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:48.220138 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e238bcf-afd3-4052-aaf0-8f91dd4bf56b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:48.221890 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e238bcf-afd3-4052-aaf0-8f91dd4bf56b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:48.232473 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1257/2531] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:48.237769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8b767-8b01-47e5-a793-ee548399d548 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:48.250124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8b767-8b01-47e5-a793-ee548399d548 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:48.255394 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc351ac-919a-476c-9499-5e8722b2903b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:48.256165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc351ac-919a-476c-9499-5e8722b2903b admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:48.256454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc351ac-919a-476c-9499-5e8722b2903b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:48.256738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc351ac-919a-476c-9499-5e8722b2903b admin admin] PATH_INFO: `/v3/projects/ec8d1f2182464a5383ef3dbac2957378/users/0e66b00cb45b46e69bb54eaafeaf85d8/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:48.258790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc351ac-919a-476c-9499-5e8722b2903b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8d1f2182464a5383ef3dbac2957378, user_id=0e66b00cb45b46e69bb54eaafeaf85d8, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:48.262482 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc351ac-919a-476c-9499-5e8722b2903b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:48.274213 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1275/2532] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:48 2022] PUT /identity/v3/projects/ec8d1f2182464a5383ef3dbac2957378/users/0e66b00cb45b46e69bb54eaafeaf85d8/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:48.288295 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a326e5-e121-479f-a3fe-2bbb0b8fadc6 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:48.288618 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a326e5-e121-479f-a3fe-2bbb0b8fadc6 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:48.288871 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a326e5-e121-479f-a3fe-2bbb0b8fadc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:48.327427 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-64a326e5-e121-479f-a3fe-2bbb0b8fadc6 None None] MFA Rules not processed for user `0e66b00cb45b46e69bb54eaafeaf85d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:48.330151 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-64a326e5-e121-479f-a3fe-2bbb0b8fadc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:48.426408 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1258/2533] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:48 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:48.439956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc351ac-919a-476c-9499-5e8722b2903b admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:48.443033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4bc351ac-919a-476c-9499-5e8722b2903b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:48.537557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc351ac-919a-476c-9499-5e8722b2903b admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:48.541318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467be1c-809d-4d23-9a6f-803d8381c1b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:48.541964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467be1c-809d-4d23-9a6f-803d8381c1b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:48.542248 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467be1c-809d-4d23-9a6f-803d8381c1b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:48.542581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467be1c-809d-4d23-9a6f-803d8381c1b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:48.552979 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467be1c-809d-4d23-9a6f-803d8381c1b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:48.554502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467be1c-809d-4d23-9a6f-803d8381c1b5 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:48.565717 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1276/2534] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:48 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:37:48.652460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a326e5-e121-479f-a3fe-2bbb0b8fadc6 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:48.667844 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a326e5-e121-479f-a3fe-2bbb0b8fadc6 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:48.672517 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0570e4a-36dd-4664-b17b-52ca31e2c23e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:48.673185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0570e4a-36dd-4664-b17b-52ca31e2c23e service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:48.677664 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0570e4a-36dd-4664-b17b-52ca31e2c23e service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:48.677964 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0570e4a-36dd-4664-b17b-52ca31e2c23e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:48.747824 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0570e4a-36dd-4664-b17b-52ca31e2c23e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:48.749333 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0570e4a-36dd-4664-b17b-52ca31e2c23e service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:48.759964 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1259/2535] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:48 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:37:50.804869 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467be1c-809d-4d23-9a6f-803d8381c1b5 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:50.806428 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3467be1c-809d-4d23-9a6f-803d8381c1b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:50.822238 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:50.899453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467be1c-809d-4d23-9a6f-803d8381c1b5 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:50.906225 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00eb76-b843-45fd-976f-8cf4ea8ba59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:50.907184 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00eb76-b843-45fd-976f-8cf4ea8ba59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:50.907580 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00eb76-b843-45fd-976f-8cf4ea8ba59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:50.908001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00eb76-b843-45fd-976f-8cf4ea8ba59d admin admin] PATH_INFO: `/v3/users/ca7c881b01e8472eb578ad8615082bde` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:50.909649 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00eb76-b843-45fd-976f-8cf4ea8ba59d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca7c881b01e8472eb578ad8615082bde)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:50.911884 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00eb76-b843-45fd-976f-8cf4ea8ba59d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:50.983369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0c00eb76-b843-45fd-976f-8cf4ea8ba59d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca7c881b01e8472eb578ad8615082bde'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:50.984093 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0c00eb76-b843-45fd-976f-8cf4ea8ba59d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca7c881b01e8472eb578ad8615082bde'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:51.000759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0c00eb76-b843-45fd-976f-8cf4ea8ba59d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca7c881b01e8472eb578ad8615082bde'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:51.001605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0c00eb76-b843-45fd-976f-8cf4ea8ba59d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca7c881b01e8472eb578ad8615082bde'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:51.010450 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-0c00eb76-b843-45fd-976f-8cf4ea8ba59d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca7c881b01e8472eb578ad8615082bde'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:51.025174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1277/2536] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:37:50 2022] DELETE /identity/v3/users/ca7c881b01e8472eb578ad8615082bde => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:51.030297 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463785f5-d818-4958-b6a0-e81d72f51419 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:51.031294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463785f5-d818-4958-b6a0-e81d72f51419 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:51.031294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463785f5-d818-4958-b6a0-e81d72f51419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:51.050536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-463785f5-d818-4958-b6a0-e81d72f51419 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:51.051786 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-463785f5-d818-4958-b6a0-e81d72f51419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:51.056318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:51.129679 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1260/2537] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:37:51.142582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00eb76-b843-45fd-976f-8cf4ea8ba59d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:51.144228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0c00eb76-b843-45fd-976f-8cf4ea8ba59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:51.243196 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00eb76-b843-45fd-976f-8cf4ea8ba59d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:51.251042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c90b6-8904-46c4-9374-751f7103ab09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:51.252382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c90b6-8904-46c4-9374-751f7103ab09 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:51.252937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c90b6-8904-46c4-9374-751f7103ab09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:51.253584 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c90b6-8904-46c4-9374-751f7103ab09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:51.272794 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c90b6-8904-46c4-9374-751f7103ab09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:51.276340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c90b6-8904-46c4-9374-751f7103ab09 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:51.298718 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1278/2538] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:37:51.639415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463785f5-d818-4958-b6a0-e81d72f51419 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:51.641214 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-463785f5-d818-4958-b6a0-e81d72f51419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:51.661108 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463785f5-d818-4958-b6a0-e81d72f51419 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:51.667009 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fa859-0a4d-4fd2-818c-25241dd24d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:51.668086 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fa859-0a4d-4fd2-818c-25241dd24d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:51.668629 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fa859-0a4d-4fd2-818c-25241dd24d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:51.668979 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fa859-0a4d-4fd2-818c-25241dd24d03 admin admin] PATH_INFO: `/v3/users/01285e1ba2514e4582c4979753efd215` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:51.670432 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466fa859-0a4d-4fd2-818c-25241dd24d03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01285e1ba2514e4582c4979753efd215)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:51.672529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466fa859-0a4d-4fd2-818c-25241dd24d03 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:51.715636 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c90b6-8904-46c4-9374-751f7103ab09 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:51.718241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ae3c90b6-8904-46c4-9374-751f7103ab09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:51.734494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c90b6-8904-46c4-9374-751f7103ab09 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:51.739641 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:51.740713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:51.741060 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:51.741442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e admin admin] PATH_INFO: `/v3/projects/78493a5f4ebf4fbaac1536ba28e0ab93` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:51.742822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e admin admin] RBAC: Authorizing `identity:delete_project(project_id=78493a5f4ebf4fbaac1536ba28e0ab93)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:51.745897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:51.755060 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-466fa859-0a4d-4fd2-818c-25241dd24d03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01285e1ba2514e4582c4979753efd215'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:51.782811 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-466fa859-0a4d-4fd2-818c-25241dd24d03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01285e1ba2514e4582c4979753efd215'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:51.798599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-466fa859-0a4d-4fd2-818c-25241dd24d03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01285e1ba2514e4582c4979753efd215'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:51.799628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-466fa859-0a4d-4fd2-818c-25241dd24d03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01285e1ba2514e4582c4979753efd215'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:51.800683 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-466fa859-0a4d-4fd2-818c-25241dd24d03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01285e1ba2514e4582c4979753efd215'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:51.803460 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:51.830019 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1261/2539] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:37:51 2022] DELETE /identity/v3/users/01285e1ba2514e4582c4979753efd215 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:51.833668 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78493a5f4ebf4fbaac1536ba28e0ab93', 'request_id': 'req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:51.835324 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa9d546-3a0b-422a-b618-7ba5dde0355a None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:51.835721 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa9d546-3a0b-422a-b618-7ba5dde0355a None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:51.836139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa9d546-3a0b-422a-b618-7ba5dde0355a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:51.842260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78493a5f4ebf4fbaac1536ba28e0ab93', 'request_id': 'req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:51.854861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78493a5f4ebf4fbaac1536ba28e0ab93', 'request_id': 'req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:51.863827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-cfa9d546-3a0b-422a-b618-7ba5dde0355a None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:51.866062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cfa9d546-3a0b-422a-b618-7ba5dde0355a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:51.879577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e admin admin] The token cache is being invalidate because project 78493a5f4ebf4fbaac1536ba28e0ab93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:37:51.880406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:51.881699 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:51.884882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1279/2540] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:37:51 2022] DELETE /identity/v3/projects/78493a5f4ebf4fbaac1536ba28e0ab93 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:51.891554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:51.894719 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:51.962280 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1262/2541] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:51.975403 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa9d546-3a0b-422a-b618-7ba5dde0355a None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:51.977796 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cfa9d546-3a0b-422a-b618-7ba5dde0355a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:52.012275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb5eb6-9104-4a60-ae7f-c19bd749e78e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:52.023053 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453e74d-845c-469e-aa24-2f7e310d4772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:52.024798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453e74d-845c-469e-aa24-2f7e310d4772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:52.025456 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453e74d-845c-469e-aa24-2f7e310d4772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:52.026560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453e74d-845c-469e-aa24-2f7e310d4772 admin admin] PATH_INFO: `/v3/users/b9ee2ce106334f6f8c06759eb8fe92e7` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:52.027515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:52.030506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453e74d-845c-469e-aa24-2f7e310d4772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9ee2ce106334f6f8c06759eb8fe92e7)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:52.035282 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453e74d-845c-469e-aa24-2f7e310d4772 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:52.087823 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa9d546-3a0b-422a-b618-7ba5dde0355a None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:52.094817 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9412d6c-4424-4397-9b3a-288f6d7d5190 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:52.096247 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9412d6c-4424-4397-9b3a-288f6d7d5190 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:52.096787 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9412d6c-4424-4397-9b3a-288f6d7d5190 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:52.097319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9412d6c-4424-4397-9b3a-288f6d7d5190 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:52.118937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9412d6c-4424-4397-9b3a-288f6d7d5190 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:52.121586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9412d6c-4424-4397-9b3a-288f6d7d5190 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:52.136557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4453e74d-845c-469e-aa24-2f7e310d4772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9ee2ce106334f6f8c06759eb8fe92e7'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:52.137492 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1263/2542] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:37:52.137825 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4453e74d-845c-469e-aa24-2f7e310d4772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9ee2ce106334f6f8c06759eb8fe92e7'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:52.167054 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4453e74d-845c-469e-aa24-2f7e310d4772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9ee2ce106334f6f8c06759eb8fe92e7'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:52.168125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4453e74d-845c-469e-aa24-2f7e310d4772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9ee2ce106334f6f8c06759eb8fe92e7'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:52.179902 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4453e74d-845c-469e-aa24-2f7e310d4772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9ee2ce106334f6f8c06759eb8fe92e7'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:52.196981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1280/2543] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:37:51 2022] DELETE /identity/v3/users/b9ee2ce106334f6f8c06759eb8fe92e7 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:52.605616 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9412d6c-4424-4397-9b3a-288f6d7d5190 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:52.626057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9412d6c-4424-4397-9b3a-288f6d7d5190 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:52.677102 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453e74d-845c-469e-aa24-2f7e310d4772 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:52.679513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4453e74d-845c-469e-aa24-2f7e310d4772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:52.744599 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be633fc-068d-4670-a9c7-03178418d83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:52.745530 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be633fc-068d-4670-a9c7-03178418d83b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:52.745818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be633fc-068d-4670-a9c7-03178418d83b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:52.746104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be633fc-068d-4670-a9c7-03178418d83b admin admin] PATH_INFO: `/v3/projects/c101660b659148f2b7a522dd22d054be` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:52.747585 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be633fc-068d-4670-a9c7-03178418d83b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c101660b659148f2b7a522dd22d054be)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:52.749582 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be633fc-068d-4670-a9c7-03178418d83b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:52.763416 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:52.809004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453e74d-845c-469e-aa24-2f7e310d4772 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:52.816460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6be633fc-068d-4670-a9c7-03178418d83b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c101660b659148f2b7a522dd22d054be', 'request_id': 'req-6be633fc-068d-4670-a9c7-03178418d83b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:52.830765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6be633fc-068d-4670-a9c7-03178418d83b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c101660b659148f2b7a522dd22d054be', 'request_id': 'req-6be633fc-068d-4670-a9c7-03178418d83b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:52.838203 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6be633fc-068d-4670-a9c7-03178418d83b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c101660b659148f2b7a522dd22d054be', 'request_id': 'req-6be633fc-068d-4670-a9c7-03178418d83b'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:52.853418 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6be633fc-068d-4670-a9c7-03178418d83b admin admin] The token cache is being invalidate because project c101660b659148f2b7a522dd22d054be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:37:52.853886 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6be633fc-068d-4670-a9c7-03178418d83b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:52.855003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6be633fc-068d-4670-a9c7-03178418d83b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:52.856819 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1264/2544] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:37:52 2022] DELETE /identity/v3/projects/c101660b659148f2b7a522dd22d054be => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:52.863063 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad559d-3943-4560-ac3c-d2a37f67b811 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:52.863504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad559d-3943-4560-ac3c-d2a37f67b811 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:52.863916 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad559d-3943-4560-ac3c-d2a37f67b811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:52.889260 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-31ad559d-3943-4560-ac3c-d2a37f67b811 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:52.891134 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-31ad559d-3943-4560-ac3c-d2a37f67b811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:52.894694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:52.896386 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:52.896899 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:52.897442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1 admin admin] PATH_INFO: `/v3/projects/25394d32b06a4a1a9accb081ac7b3f33` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:52.899227 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25394d32b06a4a1a9accb081ac7b3f33)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:52.902621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:52.902827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1265/2545] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:52.911360 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c6a53-3888-4fe8-be29-f05a20ae18c3 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:52.911654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c6a53-3888-4fe8-be29-f05a20ae18c3 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:52.911945 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c6a53-3888-4fe8-be29-f05a20ae18c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:52.933143 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-207c6a53-3888-4fe8-be29-f05a20ae18c3 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:52.934397 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-207c6a53-3888-4fe8-be29-f05a20ae18c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:52.943668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1266/2546] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:52.949191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c6a53-3888-4fe8-be29-f05a20ae18c3 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:52.963238 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c6a53-3888-4fe8-be29-f05a20ae18c3 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:52.978028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25394d32b06a4a1a9accb081ac7b3f33', 'request_id': 'req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:52.987445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25394d32b06a4a1a9accb081ac7b3f33', 'request_id': 'req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:52.999179 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25394d32b06a4a1a9accb081ac7b3f33', 'request_id': 'req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:53.020497 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:53.034852 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1 admin admin] The token cache is being invalidate because project 25394d32b06a4a1a9accb081ac7b3f33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:37:53.035437 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:53.036602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:53.039777 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1281/2547] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:37:52 2022] DELETE /identity/v3/projects/25394d32b06a4a1a9accb081ac7b3f33 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:53.042534 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873cd8e-2d00-4b3e-8aaf-6b05043e28ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:53.043341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873cd8e-2d00-4b3e-8aaf-6b05043e28ca admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:53.043738 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873cd8e-2d00-4b3e-8aaf-6b05043e28ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:53.044055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873cd8e-2d00-4b3e-8aaf-6b05043e28ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:53.044844 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873cd8e-2d00-4b3e-8aaf-6b05043e28ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:37:53.045132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873cd8e-2d00-4b3e-8aaf-6b05043e28ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:53.046379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:53.046919 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873cd8e-2d00-4b3e-8aaf-6b05043e28ca admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:53.049348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:53.066400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1267/2548] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:37:53.067745 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873cd8e-2d00-4b3e-8aaf-6b05043e28ca admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:53.070545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a873cd8e-2d00-4b3e-8aaf-6b05043e28ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:53.076646 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d6c97-645b-4a9d-a15a-3ac4f4651cb1 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:53.084540 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873cd8e-2d00-4b3e-8aaf-6b05043e28ca admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:53.086047 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961f124-95f5-4f6e-b182-f1bc5ae3cd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:53.087139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961f124-95f5-4f6e-b182-f1bc5ae3cd28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:53.087592 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961f124-95f5-4f6e-b182-f1bc5ae3cd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:53.087981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961f124-95f5-4f6e-b182-f1bc5ae3cd28 admin admin] PATH_INFO: `/v3/users/0e66b00cb45b46e69bb54eaafeaf85d8` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:53.090179 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961f124-95f5-4f6e-b182-f1bc5ae3cd28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e66b00cb45b46e69bb54eaafeaf85d8)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:53.092760 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10a27a-8bf5-47f3-b64d-32ee31dde270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:53.093588 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961f124-95f5-4f6e-b182-f1bc5ae3cd28 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:53.094052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10a27a-8bf5-47f3-b64d-32ee31dde270 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:53.094563 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10a27a-8bf5-47f3-b64d-32ee31dde270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:53.095207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10a27a-8bf5-47f3-b64d-32ee31dde270 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:53.096731 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10a27a-8bf5-47f3-b64d-32ee31dde270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:53.100491 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10a27a-8bf5-47f3-b64d-32ee31dde270 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:53.149933 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1268/2549] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:37:53 2022] POST /identity/v3/projects => generated 369 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:53.155233 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10a27a-8bf5-47f3-b64d-32ee31dde270 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:53.175713 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5961f124-95f5-4f6e-b182-f1bc5ae3cd28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e66b00cb45b46e69bb54eaafeaf85d8'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:53.176568 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5961f124-95f5-4f6e-b182-f1bc5ae3cd28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e66b00cb45b46e69bb54eaafeaf85d8'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:53.194367 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5961f124-95f5-4f6e-b182-f1bc5ae3cd28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e66b00cb45b46e69bb54eaafeaf85d8'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:53.195417 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5961f124-95f5-4f6e-b182-f1bc5ae3cd28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e66b00cb45b46e69bb54eaafeaf85d8'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:53.199502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:53.207351 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-5961f124-95f5-4f6e-b182-f1bc5ae3cd28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e66b00cb45b46e69bb54eaafeaf85d8'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:53.226005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1282/2550] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:37:53 2022] DELETE /identity/v3/users/0e66b00cb45b46e69bb54eaafeaf85d8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:53.243792 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10a27a-8bf5-47f3-b64d-32ee31dde270 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:53.316761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b677814-31c0-4895-9f03-8df9c4dcccd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:53.317529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b677814-31c0-4895-9f03-8df9c4dcccd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:53.318031 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b677814-31c0-4895-9f03-8df9c4dcccd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:53.318495 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b677814-31c0-4895-9f03-8df9c4dcccd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:53.319346 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b677814-31c0-4895-9f03-8df9c4dcccd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:53.321780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b677814-31c0-4895-9f03-8df9c4dcccd6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:53.347739 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1269/2551] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:37:53 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:37:53.353248 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961f124-95f5-4f6e-b182-f1bc5ae3cd28 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:53.356430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5961f124-95f5-4f6e-b182-f1bc5ae3cd28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:53.378268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961f124-95f5-4f6e-b182-f1bc5ae3cd28 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:53.384543 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293b6c4-6a80-4524-941f-e031f5787701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:53.385614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293b6c4-6a80-4524-941f-e031f5787701 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:53.385937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293b6c4-6a80-4524-941f-e031f5787701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:53.386313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293b6c4-6a80-4524-941f-e031f5787701 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:53.387402 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293b6c4-6a80-4524-941f-e031f5787701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:53.389574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293b6c4-6a80-4524-941f-e031f5787701 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:53.403547 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1283/2552] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:53.407881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b677814-31c0-4895-9f03-8df9c4dcccd6 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:53.420765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b677814-31c0-4895-9f03-8df9c4dcccd6 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:53.429616 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b24cd-575a-486b-a295-9fd7987f50ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:53.430378 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b24cd-575a-486b-a295-9fd7987f50ca admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:53.430772 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b24cd-575a-486b-a295-9fd7987f50ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:53.431470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b24cd-575a-486b-a295-9fd7987f50ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:53.431918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b24cd-575a-486b-a295-9fd7987f50ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:53.433857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b24cd-575a-486b-a295-9fd7987f50ca admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:53.451970 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1270/2553] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:53.456675 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293b6c4-6a80-4524-941f-e031f5787701 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:53.469186 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293b6c4-6a80-4524-941f-e031f5787701 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:53.474856 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4999010-5d54-4929-b384-12d169adaabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:53.475899 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4999010-5d54-4929-b384-12d169adaabc admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:53.476252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4999010-5d54-4929-b384-12d169adaabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:53.476779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4999010-5d54-4929-b384-12d169adaabc admin admin] PATH_INFO: `/v3/projects/67bbb62864f342deab37821b90dc10f3/users/f12823775caf49d9a9ef722b36496d23/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:53.497620 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4999010-5d54-4929-b384-12d169adaabc admin admin] RBAC: Authorizing `identity:create_grant(project_id=67bbb62864f342deab37821b90dc10f3, user_id=f12823775caf49d9a9ef722b36496d23, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:53.500775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4999010-5d54-4929-b384-12d169adaabc admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:53.520062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1284/2554] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:53 2022] PUT /identity/v3/projects/67bbb62864f342deab37821b90dc10f3/users/f12823775caf49d9a9ef722b36496d23/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:53.533540 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1762a-a290-4d40-b158-a1506cbf2b25 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:53.533834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1762a-a290-4d40-b158-a1506cbf2b25 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:53.534190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1762a-a290-4d40-b158-a1506cbf2b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:53.568833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-dbd1762a-a290-4d40-b158-a1506cbf2b25 None None] MFA Rules not processed for user `f12823775caf49d9a9ef722b36496d23`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:53.570399 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-dbd1762a-a290-4d40-b158-a1506cbf2b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:53.617631 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:53.693851 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1271/2555] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:53 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:37:53.700581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4999010-5d54-4929-b384-12d169adaabc admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:53.720678 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1762a-a290-4d40-b158-a1506cbf2b25 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:53.723378 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-dbd1762a-a290-4d40-b158-a1506cbf2b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:53.810601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4999010-5d54-4929-b384-12d169adaabc admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:53.818467 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a343d8-af42-421c-9520-60deb92e54d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:53.819746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a343d8-af42-421c-9520-60deb92e54d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:53.820168 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a343d8-af42-421c-9520-60deb92e54d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:53.820623 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a343d8-af42-421c-9520-60deb92e54d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:53.821945 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a343d8-af42-421c-9520-60deb92e54d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:53.826046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a343d8-af42-421c-9520-60deb92e54d8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:53.833093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1762a-a290-4d40-b158-a1506cbf2b25 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:53.841163 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0fcb9-f5ec-48fe-b88d-404ce2f584b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:53.842601 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0fcb9-f5ec-48fe-b88d-404ce2f584b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:53.843066 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0fcb9-f5ec-48fe-b88d-404ce2f584b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:53.843548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0fcb9-f5ec-48fe-b88d-404ce2f584b7 admin admin] PATH_INFO: `/v3/projects/ec8d1f2182464a5383ef3dbac2957378` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:53.845783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0fcb9-f5ec-48fe-b88d-404ce2f584b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec8d1f2182464a5383ef3dbac2957378)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:53.849335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0fcb9-f5ec-48fe-b88d-404ce2f584b7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:53.875281 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1285/2556] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:37:53 2022] POST /identity/v3/projects => generated 369 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:53.876973 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a343d8-af42-421c-9520-60deb92e54d8 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:53.930035 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c3e0fcb9-f5ec-48fe-b88d-404ce2f584b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec8d1f2182464a5383ef3dbac2957378', 'request_id': 'req-c3e0fcb9-f5ec-48fe-b88d-404ce2f584b7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:53.940755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c3e0fcb9-f5ec-48fe-b88d-404ce2f584b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec8d1f2182464a5383ef3dbac2957378', 'request_id': 'req-c3e0fcb9-f5ec-48fe-b88d-404ce2f584b7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:53.949326 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c3e0fcb9-f5ec-48fe-b88d-404ce2f584b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec8d1f2182464a5383ef3dbac2957378', 'request_id': 'req-c3e0fcb9-f5ec-48fe-b88d-404ce2f584b7'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:53.966064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c3e0fcb9-f5ec-48fe-b88d-404ce2f584b7 admin admin] The token cache is being invalidate because project ec8d1f2182464a5383ef3dbac2957378 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:37:53.966539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c3e0fcb9-f5ec-48fe-b88d-404ce2f584b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:53.967501 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c3e0fcb9-f5ec-48fe-b88d-404ce2f584b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:53.970025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1272/2557] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:37:53 2022] DELETE /identity/v3/projects/ec8d1f2182464a5383ef3dbac2957378 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:53.976254 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a343d8-af42-421c-9520-60deb92e54d8 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:53.977044 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68b39e-497e-4e28-aa5a-960edeba7e44 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:53.977044 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68b39e-497e-4e28-aa5a-960edeba7e44 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:53.977044 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68b39e-497e-4e28-aa5a-960edeba7e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:54.002530 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-ed68b39e-497e-4e28-aa5a-960edeba7e44 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:54.004211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ed68b39e-497e-4e28-aa5a-960edeba7e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:54.033371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:54.062163 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca0e69-03cc-4b04-b3cc-ed70745bbb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:54.063335 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca0e69-03cc-4b04-b3cc-ed70745bbb01 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:54.063829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca0e69-03cc-4b04-b3cc-ed70745bbb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:54.064315 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca0e69-03cc-4b04-b3cc-ed70745bbb01 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:54.065712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca0e69-03cc-4b04-b3cc-ed70745bbb01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:54.069950 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca0e69-03cc-4b04-b3cc-ed70745bbb01 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:54.095198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:54.105702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1286/2558] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:37:53 2022] POST /identity/v3/users => generated 291 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:37:54.110241 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca0e69-03cc-4b04-b3cc-ed70745bbb01 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:54.112148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9aca0e69-03cc-4b04-b3cc-ed70745bbb01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:54.112434 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1273/2559] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:54.122374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428b4a9-34cb-4f59-b947-9bf31fae96fa None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:54.122765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428b4a9-34cb-4f59-b947-9bf31fae96fa None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:54.123517 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428b4a9-34cb-4f59-b947-9bf31fae96fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:54.127216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca0e69-03cc-4b04-b3cc-ed70745bbb01 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:54.133001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a05c8-c3ab-40f2-9611-14321670df2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:54.133795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a05c8-c3ab-40f2-9611-14321670df2c admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:54.134150 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a05c8-c3ab-40f2-9611-14321670df2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:54.134434 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a05c8-c3ab-40f2-9611-14321670df2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:54.135994 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a05c8-c3ab-40f2-9611-14321670df2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:54.138001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a05c8-c3ab-40f2-9611-14321670df2c admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:54.151461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1287/2560] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:54.158249 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a05c8-c3ab-40f2-9611-14321670df2c admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:54.160255 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a428b4a9-34cb-4f59-b947-9bf31fae96fa None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:54.162367 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a428b4a9-34cb-4f59-b947-9bf31fae96fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:54.173458 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a05c8-c3ab-40f2-9611-14321670df2c admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:54.183449 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1274/2561] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:37:54.184652 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056a484-0f0b-4b68-aeb6-eaf103550d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:54.185829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056a484-0f0b-4b68-aeb6-eaf103550d88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:54.186182 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056a484-0f0b-4b68-aeb6-eaf103550d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:54.186532 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056a484-0f0b-4b68-aeb6-eaf103550d88 admin admin] PATH_INFO: `/v3/projects/83f83558a2bf4737863243c114069399/users/8a38ab1e0ac34496947b7c446175c2b6/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:54.189905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428b4a9-34cb-4f59-b947-9bf31fae96fa None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:54.203235 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428b4a9-34cb-4f59-b947-9bf31fae96fa None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:54.216565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c04b58-d611-4f62-898b-1aa549e4db59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:54.216820 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056a484-0f0b-4b68-aeb6-eaf103550d88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f83558a2bf4737863243c114069399, user_id=8a38ab1e0ac34496947b7c446175c2b6, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:54.217701 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c04b58-d611-4f62-898b-1aa549e4db59 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:54.218135 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c04b58-d611-4f62-898b-1aa549e4db59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:54.218941 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c04b58-d611-4f62-898b-1aa549e4db59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:54.220324 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c04b58-d611-4f62-898b-1aa549e4db59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:37:54.220835 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c04b58-d611-4f62-898b-1aa549e4db59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:54.223185 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056a484-0f0b-4b68-aeb6-eaf103550d88 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:54.224012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c04b58-d611-4f62-898b-1aa549e4db59 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:54.246858 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1288/2562] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:54 2022] PUT /identity/v3/projects/83f83558a2bf4737863243c114069399/users/8a38ab1e0ac34496947b7c446175c2b6/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:54.247605 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1275/2563] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:37:54.252722 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c04b58-d611-4f62-898b-1aa549e4db59 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:54.252989 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056a484-0f0b-4b68-aeb6-eaf103550d88 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:54.272619 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c04b58-d611-4f62-898b-1aa549e4db59 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:54.370419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056a484-0f0b-4b68-aeb6-eaf103550d88 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:54.376017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c261d-dc5a-4987-9a92-4fb4fde45112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:54.376954 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c261d-dc5a-4987-9a92-4fb4fde45112 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:54.377556 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c261d-dc5a-4987-9a92-4fb4fde45112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:54.378117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c261d-dc5a-4987-9a92-4fb4fde45112 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:54.379403 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c261d-dc5a-4987-9a92-4fb4fde45112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:54.382065 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c261d-dc5a-4987-9a92-4fb4fde45112 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:54.392847 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb2a83-8f29-415a-a4dc-88c82cfa54a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:54.394000 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb2a83-8f29-415a-a4dc-88c82cfa54a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:54.394758 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb2a83-8f29-415a-a4dc-88c82cfa54a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:54.395211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb2a83-8f29-415a-a4dc-88c82cfa54a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:54.396832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb2a83-8f29-415a-a4dc-88c82cfa54a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:54.399033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1289/2564] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:37:54.403705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb2a83-8f29-415a-a4dc-88c82cfa54a4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:54.404866 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c261d-dc5a-4987-9a92-4fb4fde45112 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:54.419933 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c261d-dc5a-4987-9a92-4fb4fde45112 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:54.425578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7000b-140e-45bd-be2b-5b9985aa0fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:54.426610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7000b-140e-45bd-be2b-5b9985aa0fba admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:54.426927 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7000b-140e-45bd-be2b-5b9985aa0fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:54.427216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7000b-140e-45bd-be2b-5b9985aa0fba admin admin] PATH_INFO: `/v3/projects/83f83558a2bf4737863243c114069399/users/8a38ab1e0ac34496947b7c446175c2b6/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:54.429601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7000b-140e-45bd-be2b-5b9985aa0fba admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f83558a2bf4737863243c114069399, user_id=8a38ab1e0ac34496947b7c446175c2b6, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:54.433210 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7000b-140e-45bd-be2b-5b9985aa0fba admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:54.447998 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1290/2565] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:54 2022] PUT /identity/v3/projects/83f83558a2bf4737863243c114069399/users/8a38ab1e0ac34496947b7c446175c2b6/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:54.453104 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1276/2566] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:37:54 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:54.459751 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7000b-140e-45bd-be2b-5b9985aa0fba admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:54.462117 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f68cf-3636-4093-816d-2dcd65e4a89e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:54.462614 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f68cf-3636-4093-816d-2dcd65e4a89e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:54.463091 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f68cf-3636-4093-816d-2dcd65e4a89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:54.517706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-ef3f68cf-3636-4093-816d-2dcd65e4a89e None None] MFA Rules not processed for user `8a38ab1e0ac34496947b7c446175c2b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:54.519054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ef3f68cf-3636-4093-816d-2dcd65e4a89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:54.551439 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7000b-140e-45bd-be2b-5b9985aa0fba admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:54.589192 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:54.638618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba011862-4da8-4a47-a1c9-d054fc03e85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:54.639275 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1277/2567] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:54 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:54.639481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba011862-4da8-4a47-a1c9-d054fc03e85a admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:54.639665 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba011862-4da8-4a47-a1c9-d054fc03e85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:54.639932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba011862-4da8-4a47-a1c9-d054fc03e85a admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:54.640785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba011862-4da8-4a47-a1c9-d054fc03e85a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:54.643014 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba011862-4da8-4a47-a1c9-d054fc03e85a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:54.656935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f68cf-3636-4093-816d-2dcd65e4a89e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:54.659235 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ef3f68cf-3636-4093-816d-2dcd65e4a89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:54.671613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1291/2568] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:37:54 2022] POST /identity/v3/users => generated 291 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:54.678042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba011862-4da8-4a47-a1c9-d054fc03e85a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:54.699408 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba011862-4da8-4a47-a1c9-d054fc03e85a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:54.706638 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dade9-2642-4c27-828b-3fa175d6b560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:54.707552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dade9-2642-4c27-828b-3fa175d6b560 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:54.707835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dade9-2642-4c27-828b-3fa175d6b560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:54.708117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dade9-2642-4c27-828b-3fa175d6b560 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:54.708870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701dade9-2642-4c27-828b-3fa175d6b560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:54.710831 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701dade9-2642-4c27-828b-3fa175d6b560 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:54.729842 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1292/2569] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:37:54.731216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dade9-2642-4c27-828b-3fa175d6b560 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:54.749585 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dade9-2642-4c27-828b-3fa175d6b560 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:54.758120 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a67fca-3adf-48ea-85fd-82a3ebc808e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:54.759424 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a67fca-3adf-48ea-85fd-82a3ebc808e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:54.760007 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a67fca-3adf-48ea-85fd-82a3ebc808e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:54.761220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a67fca-3adf-48ea-85fd-82a3ebc808e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:54.762765 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a67fca-3adf-48ea-85fd-82a3ebc808e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:54.765823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a67fca-3adf-48ea-85fd-82a3ebc808e1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:54.768358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f68cf-3636-4093-816d-2dcd65e4a89e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:54.773048 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a889b-0aeb-4ee7-99f0-7e1b2a176b92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:54.773956 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a889b-0aeb-4ee7-99f0-7e1b2a176b92 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:54.774336 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a889b-0aeb-4ee7-99f0-7e1b2a176b92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:54.775123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a889b-0aeb-4ee7-99f0-7e1b2a176b92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:54.784014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:54.793716 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1293/2570] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:37:54.794057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a889b-0aeb-4ee7-99f0-7e1b2a176b92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:54.794057 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a889b-0aeb-4ee7-99f0-7e1b2a176b92 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:54.795177 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a67fca-3adf-48ea-85fd-82a3ebc808e1 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:54.805735 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1278/2571] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:54 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:37:54.813352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a67fca-3adf-48ea-85fd-82a3ebc808e1 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:54.821729 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cc434-e43c-47cf-89ee-5638ad90cc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:54.823870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cc434-e43c-47cf-89ee-5638ad90cc45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:54.824383 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cc434-e43c-47cf-89ee-5638ad90cc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:54.824900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cc434-e43c-47cf-89ee-5638ad90cc45 admin admin] PATH_INFO: `/v3/projects/a55f91d846ba400dbc9cf6091b5caee5/users/1ed0e1b42f4843c08fb94bdc2b893159/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:54.853707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cc434-e43c-47cf-89ee-5638ad90cc45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a55f91d846ba400dbc9cf6091b5caee5, user_id=1ed0e1b42f4843c08fb94bdc2b893159, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:54.859239 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cc434-e43c-47cf-89ee-5638ad90cc45 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:54.883674 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1294/2572] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:54 2022] PUT /identity/v3/projects/a55f91d846ba400dbc9cf6091b5caee5/users/1ed0e1b42f4843c08fb94bdc2b893159/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:54.894681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a889b-0aeb-4ee7-99f0-7e1b2a176b92 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:54.896765 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2b0a889b-0aeb-4ee7-99f0-7e1b2a176b92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:54.903909 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce8a1e-6a55-4ea8-a4cf-d1f574b86b7d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:54.904378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce8a1e-6a55-4ea8-a4cf-d1f574b86b7d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:54.904834 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce8a1e-6a55-4ea8-a4cf-d1f574b86b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:54.912638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a889b-0aeb-4ee7-99f0-7e1b2a176b92 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:54.947167 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-6cce8a1e-6a55-4ea8-a4cf-d1f574b86b7d None None] MFA Rules not processed for user `1ed0e1b42f4843c08fb94bdc2b893159`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:54.949661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6cce8a1e-6a55-4ea8-a4cf-d1f574b86b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:54.988053 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591c323-ef69-45cf-b046-2f9ba5c7fc32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:54.989172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591c323-ef69-45cf-b046-2f9ba5c7fc32 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:54.989601 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591c323-ef69-45cf-b046-2f9ba5c7fc32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:54.990089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591c323-ef69-45cf-b046-2f9ba5c7fc32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:54.992616 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2591c323-ef69-45cf-b046-2f9ba5c7fc32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:55.041663 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1295/2573] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:37:55.047197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce8a1e-6a55-4ea8-a4cf-d1f574b86b7d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:55.100788 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591c323-ef69-45cf-b046-2f9ba5c7fc32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:55.103818 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591c323-ef69-45cf-b046-2f9ba5c7fc32 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:55.130005 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1279/2574] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:54 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:37:55.132269 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce8a1e-6a55-4ea8-a4cf-d1f574b86b7d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:55.138388 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1537d2-9ada-44e2-938d-45fcd0cb6059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:55.139091 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1537d2-9ada-44e2-938d-45fcd0cb6059 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:55.139371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1537d2-9ada-44e2-938d-45fcd0cb6059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:55.139648 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1537d2-9ada-44e2-938d-45fcd0cb6059 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:55.145313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1537d2-9ada-44e2-938d-45fcd0cb6059 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:55.147974 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1537d2-9ada-44e2-938d-45fcd0cb6059 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:55.153202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:55.207571 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1296/2575] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:37:55 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:55.211856 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591c323-ef69-45cf-b046-2f9ba5c7fc32 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:55.233428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591c323-ef69-45cf-b046-2f9ba5c7fc32 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:55.350312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e10a93-23d1-4af6-9482-36c124b2e281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:55.351051 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e10a93-23d1-4af6-9482-36c124b2e281 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:55.351508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e10a93-23d1-4af6-9482-36c124b2e281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:55.352494 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e10a93-23d1-4af6-9482-36c124b2e281 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:55.352732 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e10a93-23d1-4af6-9482-36c124b2e281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:55.356188 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e10a93-23d1-4af6-9482-36c124b2e281 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:55.382264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1280/2576] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:37:55 2022] POST /identity/v3/users => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:37:55.387764 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1537d2-9ada-44e2-938d-45fcd0cb6059 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:55.410015 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1537d2-9ada-44e2-938d-45fcd0cb6059 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:55.419590 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77865780-5c3f-43b6-a03b-3da9320c21b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:55.420810 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77865780-5c3f-43b6-a03b-3da9320c21b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:55.421413 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77865780-5c3f-43b6-a03b-3da9320c21b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:55.421891 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77865780-5c3f-43b6-a03b-3da9320c21b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:55.423260 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77865780-5c3f-43b6-a03b-3da9320c21b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:55.426304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77865780-5c3f-43b6-a03b-3da9320c21b9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:55.453382 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1297/2577] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:55.455105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e10a93-23d1-4af6-9482-36c124b2e281 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:55.468405 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e10a93-23d1-4af6-9482-36c124b2e281 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:55.475418 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafa2e1-e93e-4e0f-9b5d-ed3952950684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:55.476734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafa2e1-e93e-4e0f-9b5d-ed3952950684 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:55.477090 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafa2e1-e93e-4e0f-9b5d-ed3952950684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:55.477587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafa2e1-e93e-4e0f-9b5d-ed3952950684 admin admin] PATH_INFO: `/v3/projects/81335c42fa0d4c0badaccc450e272764/users/7c474da9d8454901b2fe0c85935a49b8/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:55.510354 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafa2e1-e93e-4e0f-9b5d-ed3952950684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81335c42fa0d4c0badaccc450e272764, user_id=7c474da9d8454901b2fe0c85935a49b8, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:55.515547 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafa2e1-e93e-4e0f-9b5d-ed3952950684 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:55.538117 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1281/2578] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:55 2022] PUT /identity/v3/projects/81335c42fa0d4c0badaccc450e272764/users/7c474da9d8454901b2fe0c85935a49b8/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:55.543275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77865780-5c3f-43b6-a03b-3da9320c21b9 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:55.554308 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77865780-5c3f-43b6-a03b-3da9320c21b9 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:55.641839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb0ea8-8462-47e4-9630-6a6300dcf75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:55.642711 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb0ea8-8462-47e4-9630-6a6300dcf75e admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:55.643031 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb0ea8-8462-47e4-9630-6a6300dcf75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:55.643376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb0ea8-8462-47e4-9630-6a6300dcf75e admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:55.644426 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb0ea8-8462-47e4-9630-6a6300dcf75e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:55.646407 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb0ea8-8462-47e4-9630-6a6300dcf75e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:55.659216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1298/2579] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:37:55.663472 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafa2e1-e93e-4e0f-9b5d-ed3952950684 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:55.675172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafa2e1-e93e-4e0f-9b5d-ed3952950684 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:55.680187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff56c4b-628a-4e8c-8a19-8eec0c88394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:55.681105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff56c4b-628a-4e8c-8a19-8eec0c88394f admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:55.681437 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff56c4b-628a-4e8c-8a19-8eec0c88394f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:55.681715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff56c4b-628a-4e8c-8a19-8eec0c88394f admin admin] PATH_INFO: `/v3/projects/81335c42fa0d4c0badaccc450e272764/users/7c474da9d8454901b2fe0c85935a49b8/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:55.683946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff56c4b-628a-4e8c-8a19-8eec0c88394f admin admin] RBAC: Authorizing `identity:create_grant(project_id=81335c42fa0d4c0badaccc450e272764, user_id=7c474da9d8454901b2fe0c85935a49b8, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:55.686837 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff56c4b-628a-4e8c-8a19-8eec0c88394f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:55.694539 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:55.700717 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1282/2580] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:37:55 2022] PUT /identity/v3/projects/81335c42fa0d4c0badaccc450e272764/users/7c474da9d8454901b2fe0c85935a49b8/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:37:55.712915 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175d75b-f2f3-46d7-b49f-f96ed3793c08 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:55.713201 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175d75b-f2f3-46d7-b49f-f96ed3793c08 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:55.713609 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175d75b-f2f3-46d7-b49f-f96ed3793c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:55.745492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3175d75b-f2f3-46d7-b49f-f96ed3793c08 None None] MFA Rules not processed for user `7c474da9d8454901b2fe0c85935a49b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:55.746754 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3175d75b-f2f3-46d7-b49f-f96ed3793c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:55.828587 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1299/2581] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:55.841944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff56c4b-628a-4e8c-8a19-8eec0c88394f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:55.917226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff56c4b-628a-4e8c-8a19-8eec0c88394f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:55.923285 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539191bb-6631-4851-b21b-a845ac5587d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:55.924423 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539191bb-6631-4851-b21b-a845ac5587d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:55.924856 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539191bb-6631-4851-b21b-a845ac5587d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:55.925195 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539191bb-6631-4851-b21b-a845ac5587d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:55.943256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539191bb-6631-4851-b21b-a845ac5587d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:55.946151 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539191bb-6631-4851-b21b-a845ac5587d6 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:55.964186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1283/2582] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:37:56.089779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175d75b-f2f3-46d7-b49f-f96ed3793c08 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:56.100328 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175d75b-f2f3-46d7-b49f-f96ed3793c08 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:56.104738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a57ae-e58c-49c8-9fed-1d2756880784 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:56.105561 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a57ae-e58c-49c8-9fed-1d2756880784 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:56.106007 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a57ae-e58c-49c8-9fed-1d2756880784 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:56.106352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a57ae-e58c-49c8-9fed-1d2756880784 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:56.188167 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:56.193804 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a57ae-e58c-49c8-9fed-1d2756880784 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:56.195679 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a57ae-e58c-49c8-9fed-1d2756880784 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:56.207086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1300/2583] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:37:57.229208 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02056de1-0717-4850-a488-93a9adb9a951 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:57.229208 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02056de1-0717-4850-a488-93a9adb9a951 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:57.230228 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02056de1-0717-4850-a488-93a9adb9a951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:57.264394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-02056de1-0717-4850-a488-93a9adb9a951 None None] MFA Rules not processed for user `aee77a3ed91445ab8b7f58e7f8bc348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:57.267420 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-02056de1-0717-4850-a488-93a9adb9a951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:57.394110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1284/2584] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 06:37:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:37:57.473347 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a57ae-e58c-49c8-9fed-1d2756880784 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:57.475704 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-c34a57ae-e58c-49c8-9fed-1d2756880784 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:57.586815 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a57ae-e58c-49c8-9fed-1d2756880784 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:57.595033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3c1e5-85e7-41fa-93b6-0b1d83ad3ca3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc205d053d54506bb6c9227e4a1b044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:57.595726 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3c1e5-85e7-41fa-93b6-0b1d83ad3ca3 service swift] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:57.596014 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3c1e5-85e7-41fa-93b6-0b1d83ad3ca3 service swift] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:57.596299 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3c1e5-85e7-41fa-93b6-0b1d83ad3ca3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:57.609240 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3c1e5-85e7-41fa-93b6-0b1d83ad3ca3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:57.611578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3c1e5-85e7-41fa-93b6-0b1d83ad3ca3 service swift] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:57.625512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1301/2585] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 06:37:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 06:37:59.094059 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02056de1-0717-4850-a488-93a9adb9a951 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:59.245063 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02056de1-0717-4850-a488-93a9adb9a951 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:59.254580 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394a253-d980-4681-ab3a-2a82a6f37d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:59.256275 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394a253-d980-4681-ab3a-2a82a6f37d7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:59.257154 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394a253-d980-4681-ab3a-2a82a6f37d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:59.257810 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394a253-d980-4681-ab3a-2a82a6f37d7b admin admin] PATH_INFO: `/v3/users/1ed0e1b42f4843c08fb94bdc2b893159` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:59.260697 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394a253-d980-4681-ab3a-2a82a6f37d7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ed0e1b42f4843c08fb94bdc2b893159)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:59.263887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394a253-d980-4681-ab3a-2a82a6f37d7b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:59.370067 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-1394a253-d980-4681-ab3a-2a82a6f37d7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ed0e1b42f4843c08fb94bdc2b893159'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:59.372093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:37:59.390347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-1394a253-d980-4681-ab3a-2a82a6f37d7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ed0e1b42f4843c08fb94bdc2b893159'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:59.420136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-1394a253-d980-4681-ab3a-2a82a6f37d7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ed0e1b42f4843c08fb94bdc2b893159'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:59.421110 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-1394a253-d980-4681-ab3a-2a82a6f37d7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ed0e1b42f4843c08fb94bdc2b893159'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:59.421991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-1394a253-d980-4681-ab3a-2a82a6f37d7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ed0e1b42f4843c08fb94bdc2b893159'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:59.445312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1285/2586] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:37:59 2022] DELETE /identity/v3/users/1ed0e1b42f4843c08fb94bdc2b893159 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:59.451578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604ed15-0a52-4e4e-bea8-09e15fe3a29f None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:59.451578 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604ed15-0a52-4e4e-bea8-09e15fe3a29f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:59.452235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604ed15-0a52-4e4e-bea8-09e15fe3a29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:59.479117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5604ed15-0a52-4e4e-bea8-09e15fe3a29f None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:59.480420 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5604ed15-0a52-4e4e-bea8-09e15fe3a29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:59.531581 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394a253-d980-4681-ab3a-2a82a6f37d7b admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:59.533127 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1394a253-d980-4681-ab3a-2a82a6f37d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:59.610569 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1302/2587] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:37:59.621084 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604ed15-0a52-4e4e-bea8-09e15fe3a29f None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:37:59.623982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5604ed15-0a52-4e4e-bea8-09e15fe3a29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:59.656517 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394a253-d980-4681-ab3a-2a82a6f37d7b admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:59.669436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb881cf-4f33-41ed-993c-46a952302042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:59.671349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb881cf-4f33-41ed-993c-46a952302042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:59.671698 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb881cf-4f33-41ed-993c-46a952302042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:59.672047 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb881cf-4f33-41ed-993c-46a952302042 admin admin] PATH_INFO: `/v3/users/f12823775caf49d9a9ef722b36496d23` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:59.675108 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb881cf-4f33-41ed-993c-46a952302042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f12823775caf49d9a9ef722b36496d23)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:59.679808 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb881cf-4f33-41ed-993c-46a952302042 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:59.717783 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604ed15-0a52-4e4e-bea8-09e15fe3a29f None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:37:59.724498 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85184d-2746-4a1a-893d-051e792d767d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:37:59.725762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85184d-2746-4a1a-893d-051e792d767d service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:59.726314 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85184d-2746-4a1a-893d-051e792d767d service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:59.727061 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85184d-2746-4a1a-893d-051e792d767d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:59.749618 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85184d-2746-4a1a-893d-051e792d767d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:37:59.753584 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85184d-2746-4a1a-893d-051e792d767d service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:37:59.779507 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1303/2588] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:37:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:37:59.784369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fcb881cf-4f33-41ed-993c-46a952302042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f12823775caf49d9a9ef722b36496d23'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:59.818106 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fcb881cf-4f33-41ed-993c-46a952302042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f12823775caf49d9a9ef722b36496d23'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:59.838715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fcb881cf-4f33-41ed-993c-46a952302042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f12823775caf49d9a9ef722b36496d23'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:59.841046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fcb881cf-4f33-41ed-993c-46a952302042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f12823775caf49d9a9ef722b36496d23'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:59.841988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-fcb881cf-4f33-41ed-993c-46a952302042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f12823775caf49d9a9ef722b36496d23'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:37:59.878879 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1286/2589] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:37:59 2022] DELETE /identity/v3/users/f12823775caf49d9a9ef722b36496d23 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:37:59.889311 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c76b8-ab53-4052-8ad7-cee90966beb0 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:37:59.889804 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c76b8-ab53-4052-8ad7-cee90966beb0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:37:59.890519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c76b8-ab53-4052-8ad7-cee90966beb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:37:59.928877 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-338c76b8-ab53-4052-8ad7-cee90966beb0 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:37:59.931037 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-338c76b8-ab53-4052-8ad7-cee90966beb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:37:59.979844 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:00.062258 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1304/2590] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:00.072073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb881cf-4f33-41ed-993c-46a952302042 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:00.074381 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fcb881cf-4f33-41ed-993c-46a952302042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:00.193493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:00.209072 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb881cf-4f33-41ed-993c-46a952302042 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:00.217013 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5553833-9ae9-47e1-be91-15e036fa7042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:00.219334 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5553833-9ae9-47e1-be91-15e036fa7042 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:00.220069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5553833-9ae9-47e1-be91-15e036fa7042 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:00.220582 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5553833-9ae9-47e1-be91-15e036fa7042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:00.240776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5553833-9ae9-47e1-be91-15e036fa7042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:00.249870 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5553833-9ae9-47e1-be91-15e036fa7042 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:00.267768 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1287/2591] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:38:00.435440 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c76b8-ab53-4052-8ad7-cee90966beb0 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:00.437625 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-338c76b8-ab53-4052-8ad7-cee90966beb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:00.474169 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c76b8-ab53-4052-8ad7-cee90966beb0 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:00.484217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50000f-775c-4827-93f6-66740d6c3532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:00.485598 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50000f-775c-4827-93f6-66740d6c3532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:00.487368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50000f-775c-4827-93f6-66740d6c3532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:00.487881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50000f-775c-4827-93f6-66740d6c3532 admin admin] PATH_INFO: `/v3/projects/a55f91d846ba400dbc9cf6091b5caee5` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:00.489704 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50000f-775c-4827-93f6-66740d6c3532 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a55f91d846ba400dbc9cf6091b5caee5)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:00.492773 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50000f-775c-4827-93f6-66740d6c3532 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:00.582165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-fb50000f-775c-4827-93f6-66740d6c3532 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a55f91d846ba400dbc9cf6091b5caee5', 'request_id': 'req-fb50000f-775c-4827-93f6-66740d6c3532'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:00.591779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-fb50000f-775c-4827-93f6-66740d6c3532 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a55f91d846ba400dbc9cf6091b5caee5', 'request_id': 'req-fb50000f-775c-4827-93f6-66740d6c3532'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:00.606562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-fb50000f-775c-4827-93f6-66740d6c3532 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a55f91d846ba400dbc9cf6091b5caee5', 'request_id': 'req-fb50000f-775c-4827-93f6-66740d6c3532'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:00.641047 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-fb50000f-775c-4827-93f6-66740d6c3532 admin admin] The token cache is being invalidate because project a55f91d846ba400dbc9cf6091b5caee5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:38:00.643729 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-fb50000f-775c-4827-93f6-66740d6c3532 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:00.645187 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-fb50000f-775c-4827-93f6-66740d6c3532 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:00.648659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1305/2592] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:38:00 2022] DELETE /identity/v3/projects/a55f91d846ba400dbc9cf6091b5caee5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:00.653416 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5553833-9ae9-47e1-be91-15e036fa7042 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:00.671077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5553833-9ae9-47e1-be91-15e036fa7042 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:00.756129 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50000f-775c-4827-93f6-66740d6c3532 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:00.757673 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fb50000f-775c-4827-93f6-66740d6c3532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:00.791918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f454e2-3271-41be-bbf9-f2e43924248c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:00.793520 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f454e2-3271-41be-bbf9-f2e43924248c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:00.793991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f454e2-3271-41be-bbf9-f2e43924248c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:00.794702 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f454e2-3271-41be-bbf9-f2e43924248c admin admin] PATH_INFO: `/v3/users/7c474da9d8454901b2fe0c85935a49b8` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:00.797487 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f454e2-3271-41be-bbf9-f2e43924248c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c474da9d8454901b2fe0c85935a49b8)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:00.806171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f454e2-3271-41be-bbf9-f2e43924248c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:00.899966 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-13f454e2-3271-41be-bbf9-f2e43924248c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c474da9d8454901b2fe0c85935a49b8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:00.931957 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-13f454e2-3271-41be-bbf9-f2e43924248c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c474da9d8454901b2fe0c85935a49b8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:00.943609 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-13f454e2-3271-41be-bbf9-f2e43924248c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c474da9d8454901b2fe0c85935a49b8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:00.944306 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-13f454e2-3271-41be-bbf9-f2e43924248c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c474da9d8454901b2fe0c85935a49b8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:00.945130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-13f454e2-3271-41be-bbf9-f2e43924248c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c474da9d8454901b2fe0c85935a49b8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:00.955946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50000f-775c-4827-93f6-66740d6c3532 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:00.966218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1288/2593] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:38:00 2022] DELETE /identity/v3/users/7c474da9d8454901b2fe0c85935a49b8 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:00.967251 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edebc3-9b6a-4a50-8cec-cdae446f2274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:00.969511 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edebc3-9b6a-4a50-8cec-cdae446f2274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:00.970036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edebc3-9b6a-4a50-8cec-cdae446f2274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:00.970394 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edebc3-9b6a-4a50-8cec-cdae446f2274 admin admin] PATH_INFO: `/v3/projects/67bbb62864f342deab37821b90dc10f3` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:00.972724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edebc3-9b6a-4a50-8cec-cdae446f2274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67bbb62864f342deab37821b90dc10f3)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:00.976493 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edebc3-9b6a-4a50-8cec-cdae446f2274 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:01.094902 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a2edebc3-9b6a-4a50-8cec-cdae446f2274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67bbb62864f342deab37821b90dc10f3', 'request_id': 'req-a2edebc3-9b6a-4a50-8cec-cdae446f2274'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:01.098002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:01.108293 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a2edebc3-9b6a-4a50-8cec-cdae446f2274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67bbb62864f342deab37821b90dc10f3', 'request_id': 'req-a2edebc3-9b6a-4a50-8cec-cdae446f2274'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:01.121838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a2edebc3-9b6a-4a50-8cec-cdae446f2274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67bbb62864f342deab37821b90dc10f3', 'request_id': 'req-a2edebc3-9b6a-4a50-8cec-cdae446f2274'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:01.148861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a2edebc3-9b6a-4a50-8cec-cdae446f2274 admin admin] The token cache is being invalidate because project 67bbb62864f342deab37821b90dc10f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:38:01.148861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a2edebc3-9b6a-4a50-8cec-cdae446f2274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:01.148861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a2edebc3-9b6a-4a50-8cec-cdae446f2274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:01.155443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1306/2594] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:38:00 2022] DELETE /identity/v3/projects/67bbb62864f342deab37821b90dc10f3 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:01.158077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f454e2-3271-41be-bbf9-f2e43924248c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:01.161099 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-13f454e2-3271-41be-bbf9-f2e43924248c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:01.296565 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f454e2-3271-41be-bbf9-f2e43924248c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:01.334425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:01.405829 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edebc3-9b6a-4a50-8cec-cdae446f2274 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:01.407493 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a2edebc3-9b6a-4a50-8cec-cdae446f2274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:01.450115 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e99b4-c240-411f-86f3-7f7c4251428e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:01.456985 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e99b4-c240-411f-86f3-7f7c4251428e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:01.456985 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e99b4-c240-411f-86f3-7f7c4251428e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:01.456985 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e99b4-c240-411f-86f3-7f7c4251428e admin admin] PATH_INFO: `/v3/users/8a38ab1e0ac34496947b7c446175c2b6` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:01.459336 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e99b4-c240-411f-86f3-7f7c4251428e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a38ab1e0ac34496947b7c446175c2b6)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:01.462820 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e99b4-c240-411f-86f3-7f7c4251428e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:01.541067 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edebc3-9b6a-4a50-8cec-cdae446f2274 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:01.548680 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3481d0a-ba56-4b2d-b157-545b600b64e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:01.549838 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3481d0a-ba56-4b2d-b157-545b600b64e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:01.550196 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3481d0a-ba56-4b2d-b157-545b600b64e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:01.550626 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3481d0a-ba56-4b2d-b157-545b600b64e1 admin admin] PATH_INFO: `/v3/projects/81335c42fa0d4c0badaccc450e272764` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:01.552569 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3481d0a-ba56-4b2d-b157-545b600b64e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81335c42fa0d4c0badaccc450e272764)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:01.557527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3481d0a-ba56-4b2d-b157-545b600b64e1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:01.574227 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5d9e99b4-c240-411f-86f3-7f7c4251428e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a38ab1e0ac34496947b7c446175c2b6'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:01.601613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5d9e99b4-c240-411f-86f3-7f7c4251428e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a38ab1e0ac34496947b7c446175c2b6'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:01.623561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5d9e99b4-c240-411f-86f3-7f7c4251428e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a38ab1e0ac34496947b7c446175c2b6'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:01.624386 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5d9e99b4-c240-411f-86f3-7f7c4251428e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a38ab1e0ac34496947b7c446175c2b6'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:01.625515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5d9e99b4-c240-411f-86f3-7f7c4251428e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a38ab1e0ac34496947b7c446175c2b6'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:01.646554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1289/2595] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:38:01 2022] DELETE /identity/v3/users/8a38ab1e0ac34496947b7c446175c2b6 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:01.688299 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b3481d0a-ba56-4b2d-b157-545b600b64e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81335c42fa0d4c0badaccc450e272764', 'request_id': 'req-b3481d0a-ba56-4b2d-b157-545b600b64e1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:01.699355 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b3481d0a-ba56-4b2d-b157-545b600b64e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81335c42fa0d4c0badaccc450e272764', 'request_id': 'req-b3481d0a-ba56-4b2d-b157-545b600b64e1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:01.715852 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b3481d0a-ba56-4b2d-b157-545b600b64e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81335c42fa0d4c0badaccc450e272764', 'request_id': 'req-b3481d0a-ba56-4b2d-b157-545b600b64e1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:01.737642 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b3481d0a-ba56-4b2d-b157-545b600b64e1 admin admin] The token cache is being invalidate because project 81335c42fa0d4c0badaccc450e272764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:38:01.737978 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b3481d0a-ba56-4b2d-b157-545b600b64e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:01.738649 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b3481d0a-ba56-4b2d-b157-545b600b64e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:01.740751 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1307/2596] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:38:01 2022] DELETE /identity/v3/projects/81335c42fa0d4c0badaccc450e272764 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:01.750837 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba968d-34d0-4ccb-93ba-be7074ff044e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:01.752222 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba968d-34d0-4ccb-93ba-be7074ff044e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:01.752751 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba968d-34d0-4ccb-93ba-be7074ff044e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:01.786260 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4aba968d-34d0-4ccb-93ba-be7074ff044e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:01.787671 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4aba968d-34d0-4ccb-93ba-be7074ff044e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:01.888557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1290/2597] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:01.894851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e453bfd-46c9-4efd-a1ad-bf5137156ea0 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:01.895326 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e453bfd-46c9-4efd-a1ad-bf5137156ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:01.896376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e453bfd-46c9-4efd-a1ad-bf5137156ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:01.920707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3e453bfd-46c9-4efd-a1ad-bf5137156ea0 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:01.922108 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3e453bfd-46c9-4efd-a1ad-bf5137156ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:01.936012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1308/2598] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:01.937801 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba968d-34d0-4ccb-93ba-be7074ff044e None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:01.968672 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba968d-34d0-4ccb-93ba-be7074ff044e None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:01.978637 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a959c75-7467-4249-a5a0-954b17a32e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:01.980806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a959c75-7467-4249-a5a0-954b17a32e7f admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:01.981379 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a959c75-7467-4249-a5a0-954b17a32e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:01.981889 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a959c75-7467-4249-a5a0-954b17a32e7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:01.982936 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a959c75-7467-4249-a5a0-954b17a32e7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:38:01.983255 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a959c75-7467-4249-a5a0-954b17a32e7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:01.985001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a959c75-7467-4249-a5a0-954b17a32e7f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:02.001624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1291/2599] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:38:02.007145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e453bfd-46c9-4efd-a1ad-bf5137156ea0 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:02.025392 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e453bfd-46c9-4efd-a1ad-bf5137156ea0 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:02.033817 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06cb8b-80e8-4eb1-aee6-7451a5f7bf84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:02.034802 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06cb8b-80e8-4eb1-aee6-7451a5f7bf84 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:02.035183 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06cb8b-80e8-4eb1-aee6-7451a5f7bf84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:02.035506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06cb8b-80e8-4eb1-aee6-7451a5f7bf84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:02.036448 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06cb8b-80e8-4eb1-aee6-7451a5f7bf84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:02.039058 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06cb8b-80e8-4eb1-aee6-7451a5f7bf84 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:02.095874 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1309/2600] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:38:02 2022] POST /identity/v3/projects => generated 389 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:38:02.100827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a959c75-7467-4249-a5a0-954b17a32e7f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:02.109353 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06cb8b-80e8-4eb1-aee6-7451a5f7bf84 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:02.111775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ba06cb8b-80e8-4eb1-aee6-7451a5f7bf84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:02.127310 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a959c75-7467-4249-a5a0-954b17a32e7f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:02.172557 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:02.208427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:02.258488 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65defd6-a7da-4d7f-81a3-ee2920ae179e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:02.259613 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65defd6-a7da-4d7f-81a3-ee2920ae179e admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:02.261747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65defd6-a7da-4d7f-81a3-ee2920ae179e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:02.262532 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65defd6-a7da-4d7f-81a3-ee2920ae179e admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:02.263552 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65defd6-a7da-4d7f-81a3-ee2920ae179e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:02.263779 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06cb8b-80e8-4eb1-aee6-7451a5f7bf84 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:02.265986 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65defd6-a7da-4d7f-81a3-ee2920ae179e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:02.283114 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f867e2-5395-4370-ac97-1b2f0d1b6137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:02.284446 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f867e2-5395-4370-ac97-1b2f0d1b6137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:02.284813 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f867e2-5395-4370-ac97-1b2f0d1b6137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:02.285107 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f867e2-5395-4370-ac97-1b2f0d1b6137 admin admin] PATH_INFO: `/v3/projects/83f83558a2bf4737863243c114069399` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:02.286627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f867e2-5395-4370-ac97-1b2f0d1b6137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83f83558a2bf4737863243c114069399)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:02.291035 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f867e2-5395-4370-ac97-1b2f0d1b6137 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:02.298028 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1292/2601] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:38:02 2022] POST /identity/v3/users => generated 295 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:38:02.304479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65defd6-a7da-4d7f-81a3-ee2920ae179e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:02.328600 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65defd6-a7da-4d7f-81a3-ee2920ae179e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:02.335948 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7670d-7d6b-4f10-8f38-30c4907d51d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:02.336901 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7670d-7d6b-4f10-8f38-30c4907d51d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:02.337496 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7670d-7d6b-4f10-8f38-30c4907d51d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:02.338186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7670d-7d6b-4f10-8f38-30c4907d51d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:02.340061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7670d-7d6b-4f10-8f38-30c4907d51d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:02.342036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7670d-7d6b-4f10-8f38-30c4907d51d4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:02.355649 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1293/2602] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:38:02.360307 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7670d-7d6b-4f10-8f38-30c4907d51d4 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:02.381563 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7670d-7d6b-4f10-8f38-30c4907d51d4 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:02.407344 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b9f867e2-5395-4370-ac97-1b2f0d1b6137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83f83558a2bf4737863243c114069399', 'request_id': 'req-b9f867e2-5395-4370-ac97-1b2f0d1b6137'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:02.421588 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b9f867e2-5395-4370-ac97-1b2f0d1b6137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83f83558a2bf4737863243c114069399', 'request_id': 'req-b9f867e2-5395-4370-ac97-1b2f0d1b6137'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:02.438577 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b9f867e2-5395-4370-ac97-1b2f0d1b6137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83f83558a2bf4737863243c114069399', 'request_id': 'req-b9f867e2-5395-4370-ac97-1b2f0d1b6137'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:02.472405 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b9f867e2-5395-4370-ac97-1b2f0d1b6137 admin admin] The token cache is being invalidate because project 83f83558a2bf4737863243c114069399 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:38:02.472774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b9f867e2-5395-4370-ac97-1b2f0d1b6137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:02.473599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b9f867e2-5395-4370-ac97-1b2f0d1b6137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:02.478952 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1310/2603] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:38:02 2022] DELETE /identity/v3/projects/83f83558a2bf4737863243c114069399 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:38:02.480911 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27db3c-7929-4d59-b261-f0cd0df7e7b5 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:02.481252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27db3c-7929-4d59-b261-f0cd0df7e7b5 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:02.481524 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27db3c-7929-4d59-b261-f0cd0df7e7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:02.503842 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132efa0c-ef59-4880-892b-04e7387fc423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:02.504588 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132efa0c-ef59-4880-892b-04e7387fc423 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:02.504972 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132efa0c-ef59-4880-892b-04e7387fc423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:02.505807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132efa0c-ef59-4880-892b-04e7387fc423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:02.506108 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-8c27db3c-7929-4d59-b261-f0cd0df7e7b5 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:02.506682 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132efa0c-ef59-4880-892b-04e7387fc423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:02.507542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8c27db3c-7929-4d59-b261-f0cd0df7e7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:02.509102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132efa0c-ef59-4880-892b-04e7387fc423 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:02.519543 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1311/2604] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:02.531569 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1294/2605] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:38:02.534598 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923c602-8c75-4938-a5b5-ad3a691dcc18 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:02.535118 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923c602-8c75-4938-a5b5-ad3a691dcc18 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:02.535382 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132efa0c-ef59-4880-892b-04e7387fc423 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:02.535637 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923c602-8c75-4938-a5b5-ad3a691dcc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:02.537399 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-132efa0c-ef59-4880-892b-04e7387fc423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:02.561437 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132efa0c-ef59-4880-892b-04e7387fc423 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:02.578211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-2923c602-8c75-4938-a5b5-ad3a691dcc18 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:02.580271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2923c602-8c75-4938-a5b5-ad3a691dcc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:02.581921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4b7bf-175c-42ae-a1ca-6b787ef91bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:02.584051 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4b7bf-175c-42ae-a1ca-6b787ef91bb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:02.584051 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4b7bf-175c-42ae-a1ca-6b787ef91bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:02.585321 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4b7bf-175c-42ae-a1ca-6b787ef91bb7 admin admin] PATH_INFO: `/v3/projects/dbf45826ee9545a982bd3e7a83a423fd/users/44f4d15284fa4a53906c2efca2d1f453/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:02.596091 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1312/2606] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:02.600921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923c602-8c75-4938-a5b5-ad3a691dcc18 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:02.612362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923c602-8c75-4938-a5b5-ad3a691dcc18 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:02.619316 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155484c5-c3b9-4420-a331-b2c8b518d531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:02.620766 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155484c5-c3b9-4420-a331-b2c8b518d531 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:02.621053 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155484c5-c3b9-4420-a331-b2c8b518d531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:02.621471 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155484c5-c3b9-4420-a331-b2c8b518d531 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:02.622305 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155484c5-c3b9-4420-a331-b2c8b518d531 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:38:02.622656 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155484c5-c3b9-4420-a331-b2c8b518d531 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:02.623256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba4b7bf-175c-42ae-a1ca-6b787ef91bb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf45826ee9545a982bd3e7a83a423fd, user_id=44f4d15284fa4a53906c2efca2d1f453, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:02.624622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155484c5-c3b9-4420-a331-b2c8b518d531 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:02.632187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba4b7bf-175c-42ae-a1ca-6b787ef91bb7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:02.641127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1313/2607] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:38:02.645909 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155484c5-c3b9-4420-a331-b2c8b518d531 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:02.665419 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1295/2608] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:38:02 2022] PUT /identity/v3/projects/dbf45826ee9545a982bd3e7a83a423fd/users/44f4d15284fa4a53906c2efca2d1f453/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:38:02.665693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155484c5-c3b9-4420-a331-b2c8b518d531 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:02.674066 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e0170-b615-412f-851b-b4e3afc88bfa None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:02.675179 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e0170-b615-412f-851b-b4e3afc88bfa None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:02.676099 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e0170-b615-412f-851b-b4e3afc88bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:02.732654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c19e0170-b615-412f-851b-b4e3afc88bfa None None] MFA Rules not processed for user `44f4d15284fa4a53906c2efca2d1f453`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:02.733964 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c19e0170-b615-412f-851b-b4e3afc88bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:02.812522 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b93e72-683d-4f53-899f-eb2e84d49237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:02.813125 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b93e72-683d-4f53-899f-eb2e84d49237 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:02.819634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b93e72-683d-4f53-899f-eb2e84d49237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:02.819634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b93e72-683d-4f53-899f-eb2e84d49237 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:02.820457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b93e72-683d-4f53-899f-eb2e84d49237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:02.822508 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b93e72-683d-4f53-899f-eb2e84d49237 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:02.882402 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1296/2609] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:02.887247 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e0170-b615-412f-851b-b4e3afc88bfa None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:02.899655 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1314/2610] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:38:02 2022] POST /identity/v3/projects => generated 385 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:38:02.906237 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b93e72-683d-4f53-899f-eb2e84d49237 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:02.907668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:02.937080 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e0170-b615-412f-851b-b4e3afc88bfa None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:03.005607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30184a5-b450-49ac-b70c-931f2d3319fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:03.006286 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30184a5-b450-49ac-b70c-931f2d3319fc admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:03.006750 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30184a5-b450-49ac-b70c-931f2d3319fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:03.007043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30184a5-b450-49ac-b70c-931f2d3319fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:03.007916 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30184a5-b450-49ac-b70c-931f2d3319fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:03.010003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30184a5-b450-49ac-b70c-931f2d3319fc admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:03.022936 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:03.051737 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1297/2611] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:38:02 2022] POST /identity/v3/projects => generated 383 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:38:03.055922 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30184a5-b450-49ac-b70c-931f2d3319fc admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:03.086233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b93e72-683d-4f53-899f-eb2e84d49237 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:03.158169 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30184a5-b450-49ac-b70c-931f2d3319fc admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:03.165551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a2268-4f6b-4244-a91f-cc42288eef4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:03.167914 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a2268-4f6b-4244-a91f-cc42288eef4d admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:03.168650 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a2268-4f6b-4244-a91f-cc42288eef4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:03.169144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a2268-4f6b-4244-a91f-cc42288eef4d admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:03.170990 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a2268-4f6b-4244-a91f-cc42288eef4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:03.174140 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a2268-4f6b-4244-a91f-cc42288eef4d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:03.225981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cbb78-fc55-4430-9daf-cc28064e94e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:03.227042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cbb78-fc55-4430-9daf-cc28064e94e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:03.227520 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cbb78-fc55-4430-9daf-cc28064e94e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:03.228015 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cbb78-fc55-4430-9daf-cc28064e94e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:03.230728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343cbb78-fc55-4430-9daf-cc28064e94e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:03.233393 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1298/2612] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:38:03 2022] POST /identity/v3/users => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:38:03.234901 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343cbb78-fc55-4430-9daf-cc28064e94e0 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:03.237091 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a2268-4f6b-4244-a91f-cc42288eef4d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:03.264123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a2268-4f6b-4244-a91f-cc42288eef4d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:03.264935 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1315/2613] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:38:02 2022] POST /identity/v3/users => generated 293 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:38:03.269006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cbb78-fc55-4430-9daf-cc28064e94e0 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:03.273192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93926b9a-1c24-4654-9451-5dc4e9dd9857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:03.274834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93926b9a-1c24-4654-9451-5dc4e9dd9857 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:03.276543 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93926b9a-1c24-4654-9451-5dc4e9dd9857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:03.278069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93926b9a-1c24-4654-9451-5dc4e9dd9857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:03.279340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93926b9a-1c24-4654-9451-5dc4e9dd9857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:03.282601 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93926b9a-1c24-4654-9451-5dc4e9dd9857 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:03.289763 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cbb78-fc55-4430-9daf-cc28064e94e0 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:03.307607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1299/2614] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:38:03.308801 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d4a0a-6fc8-4e5e-a84f-903d7004ffe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:03.310372 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93926b9a-1c24-4654-9451-5dc4e9dd9857 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:03.311153 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d4a0a-6fc8-4e5e-a84f-903d7004ffe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:03.311439 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d4a0a-6fc8-4e5e-a84f-903d7004ffe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:03.311725 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d4a0a-6fc8-4e5e-a84f-903d7004ffe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:03.312493 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d4a0a-6fc8-4e5e-a84f-903d7004ffe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:03.323693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d4a0a-6fc8-4e5e-a84f-903d7004ffe0 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:03.340690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93926b9a-1c24-4654-9451-5dc4e9dd9857 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:03.351508 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1316/2615] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:38:03.353495 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66582f5-7ba1-46f1-84fd-ccbb81a7f2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:03.355752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66582f5-7ba1-46f1-84fd-ccbb81a7f2cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:03.356393 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66582f5-7ba1-46f1-84fd-ccbb81a7f2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:03.356983 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66582f5-7ba1-46f1-84fd-ccbb81a7f2cc admin admin] PATH_INFO: `/v3/projects/4ea15701a40a4a789cdf72038a5e2456/users/aac7a1fc1d0e4228a6c96f6bca0d2673/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:03.357621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d4a0a-6fc8-4e5e-a84f-903d7004ffe0 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:03.376757 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d4a0a-6fc8-4e5e-a84f-903d7004ffe0 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:03.385491 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c39b6-776c-4e29-823d-9da88b752426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:03.387133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c39b6-776c-4e29-823d-9da88b752426 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:03.387843 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c39b6-776c-4e29-823d-9da88b752426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:03.388522 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c39b6-776c-4e29-823d-9da88b752426 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:03.389870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c39b6-776c-4e29-823d-9da88b752426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:03.390759 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66582f5-7ba1-46f1-84fd-ccbb81a7f2cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea15701a40a4a789cdf72038a5e2456, user_id=aac7a1fc1d0e4228a6c96f6bca0d2673, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:03.393180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c39b6-776c-4e29-823d-9da88b752426 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:03.396464 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66582f5-7ba1-46f1-84fd-ccbb81a7f2cc admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:03.421621 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1317/2616] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:38:03.423488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c39b6-776c-4e29-823d-9da88b752426 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:03.438340 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1300/2617] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:38:03 2022] PUT /identity/v3/projects/4ea15701a40a4a789cdf72038a5e2456/users/aac7a1fc1d0e4228a6c96f6bca0d2673/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:03.443736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66582f5-7ba1-46f1-84fd-ccbb81a7f2cc admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:03.446181 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c39b6-776c-4e29-823d-9da88b752426 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:03.525943 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ad363-a1f6-4e4f-a18f-97be7c7c5827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:03.527598 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ad363-a1f6-4e4f-a18f-97be7c7c5827 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:03.527909 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ad363-a1f6-4e4f-a18f-97be7c7c5827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:03.528167 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ad363-a1f6-4e4f-a18f-97be7c7c5827 admin admin] PATH_INFO: `/v3/projects/198b9677b0a04e0ead76704f5c5b1aba/users/17e0316f7ea74eeeb675650cf6f6cfe6/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:03.546702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78ad363-a1f6-4e4f-a18f-97be7c7c5827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=198b9677b0a04e0ead76704f5c5b1aba, user_id=17e0316f7ea74eeeb675650cf6f6cfe6, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:03.549625 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78ad363-a1f6-4e4f-a18f-97be7c7c5827 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:03.571597 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1318/2618] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:38:03 2022] PUT /identity/v3/projects/198b9677b0a04e0ead76704f5c5b1aba/users/17e0316f7ea74eeeb675650cf6f6cfe6/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:03.583115 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:03.585328 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e8969-3193-4c7f-9ebd-fbeaa5d40a10 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:03.586097 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e8969-3193-4c7f-9ebd-fbeaa5d40a10 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:03.586097 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e8969-3193-4c7f-9ebd-fbeaa5d40a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:03.631271 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-018e8969-3193-4c7f-9ebd-fbeaa5d40a10 None None] MFA Rules not processed for user `17e0316f7ea74eeeb675650cf6f6cfe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:03.633350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-018e8969-3193-4c7f-9ebd-fbeaa5d40a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:03.637253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66582f5-7ba1-46f1-84fd-ccbb81a7f2cc admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:03.666794 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:03.763692 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1319/2619] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:03.768100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e8969-3193-4c7f-9ebd-fbeaa5d40a10 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:03.775793 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0761a89-a0ce-4e1d-b9bf-572b3e443fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:03.777790 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0761a89-a0ce-4e1d-b9bf-572b3e443fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:03.778305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0761a89-a0ce-4e1d-b9bf-572b3e443fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:03.778806 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0761a89-a0ce-4e1d-b9bf-572b3e443fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:03.780001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0761a89-a0ce-4e1d-b9bf-572b3e443fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:03.782079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0761a89-a0ce-4e1d-b9bf-572b3e443fb1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:03.806916 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1301/2620] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:38:03.813852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0761a89-a0ce-4e1d-b9bf-572b3e443fb1 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:03.829882 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0761a89-a0ce-4e1d-b9bf-572b3e443fb1 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:03.836425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea64c6-6951-4571-aeeb-815349fd731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:03.837781 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea64c6-6951-4571-aeeb-815349fd731b admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:03.838239 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea64c6-6951-4571-aeeb-815349fd731b admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:03.839228 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea64c6-6951-4571-aeeb-815349fd731b admin admin] PATH_INFO: `/v3/projects/4ea15701a40a4a789cdf72038a5e2456/users/aac7a1fc1d0e4228a6c96f6bca0d2673/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:03.842655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea64c6-6951-4571-aeeb-815349fd731b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea15701a40a4a789cdf72038a5e2456, user_id=aac7a1fc1d0e4228a6c96f6bca0d2673, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:03.846371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea64c6-6951-4571-aeeb-815349fd731b admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:03.863796 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1302/2621] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:38:03 2022] PUT /identity/v3/projects/4ea15701a40a4a789cdf72038a5e2456/users/aac7a1fc1d0e4228a6c96f6bca0d2673/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:03.888439 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5ebeb-8cf3-4f85-a062-012037cad248 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:03.888736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5ebeb-8cf3-4f85-a062-012037cad248 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:03.889002 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5ebeb-8cf3-4f85-a062-012037cad248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:03.941318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e8969-3193-4c7f-9ebd-fbeaa5d40a10 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:03.944288 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-50a5ebeb-8cf3-4f85-a062-012037cad248 None None] MFA Rules not processed for user `aac7a1fc1d0e4228a6c96f6bca0d2673`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:03.945798 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-50a5ebeb-8cf3-4f85-a062-012037cad248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:04.077566 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1303/2622] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:04.097640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5ebeb-8cf3-4f85-a062-012037cad248 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:04.098078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7ead3-b859-4d1a-a9f5-ee0d9f3d9ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:04.099044 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-50a5ebeb-8cf3-4f85-a062-012037cad248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:04.099413 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7ead3-b859-4d1a-a9f5-ee0d9f3d9ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:04.099931 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7ead3-b859-4d1a-a9f5-ee0d9f3d9ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:04.100485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7ead3-b859-4d1a-a9f5-ee0d9f3d9ec4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:04.101976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7ead3-b859-4d1a-a9f5-ee0d9f3d9ec4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:04.107835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7ead3-b859-4d1a-a9f5-ee0d9f3d9ec4 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:04.171212 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1320/2623] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:38:03 2022] POST /identity/v3/projects => generated 387 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:38:04.173202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7ead3-b859-4d1a-a9f5-ee0d9f3d9ec4 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:04.204228 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5ebeb-8cf3-4f85-a062-012037cad248 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:04.219515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:04.296081 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e72616-8ca8-471c-90d6-3ca7bffc284f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:04.296865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e72616-8ca8-471c-90d6-3ca7bffc284f service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:04.297160 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e72616-8ca8-471c-90d6-3ca7bffc284f service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:04.297571 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e72616-8ca8-471c-90d6-3ca7bffc284f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:04.308129 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:04.358329 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7ead3-b859-4d1a-a9f5-ee0d9f3d9ec4 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:04.381629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b63350-9b67-4d58-8323-31ffeccae79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:04.382799 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b63350-9b67-4d58-8323-31ffeccae79d admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:04.383286 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b63350-9b67-4d58-8323-31ffeccae79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:04.383798 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b63350-9b67-4d58-8323-31ffeccae79d admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:04.386202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b63350-9b67-4d58-8323-31ffeccae79d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:04.390392 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b63350-9b67-4d58-8323-31ffeccae79d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:04.393548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e72616-8ca8-471c-90d6-3ca7bffc284f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:04.397262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e72616-8ca8-471c-90d6-3ca7bffc284f service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:04.423908 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1304/2624] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:38:04.440816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1321/2625] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:38:04 2022] POST /identity/v3/users => generated 300 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:38:04.441992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e72616-8ca8-471c-90d6-3ca7bffc284f service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:04.465435 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e72616-8ca8-471c-90d6-3ca7bffc284f service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:04.474938 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fca32b-1e1e-45a8-926b-5774098b4327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:04.475629 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fca32b-1e1e-45a8-926b-5774098b4327 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:04.475917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fca32b-1e1e-45a8-926b-5774098b4327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:04.476199 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fca32b-1e1e-45a8-926b-5774098b4327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:04.476964 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fca32b-1e1e-45a8-926b-5774098b4327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:04.481271 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fca32b-1e1e-45a8-926b-5774098b4327 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:04.509344 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1305/2626] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:38:04.516228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b63350-9b67-4d58-8323-31ffeccae79d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:04.535812 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b63350-9b67-4d58-8323-31ffeccae79d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:04.547460 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71a6a4-0cdd-4e19-9df3-18f01a60c9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:04.549427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71a6a4-0cdd-4e19-9df3-18f01a60c9ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:04.550440 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71a6a4-0cdd-4e19-9df3-18f01a60c9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:04.551500 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71a6a4-0cdd-4e19-9df3-18f01a60c9ad admin admin] PATH_INFO: `/v3/projects/17297f6951334b9f9ebd75da1e64eb0f/users/9db75b86ddc04f4e93e7bb8d9674d506/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:04.593381 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71a6a4-0cdd-4e19-9df3-18f01a60c9ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=17297f6951334b9f9ebd75da1e64eb0f, user_id=9db75b86ddc04f4e93e7bb8d9674d506, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:04.596960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71a6a4-0cdd-4e19-9df3-18f01a60c9ad admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:04.609218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fca32b-1e1e-45a8-926b-5774098b4327 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:04.617740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1322/2627] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:38:04 2022] PUT /identity/v3/projects/17297f6951334b9f9ebd75da1e64eb0f/users/9db75b86ddc04f4e93e7bb8d9674d506/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:04.629537 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71a6a4-0cdd-4e19-9df3-18f01a60c9ad admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:04.638869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fca32b-1e1e-45a8-926b-5774098b4327 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:04.750846 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013dbe88-f97c-42a9-8c0e-deeb21217c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:04.751803 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013dbe88-f97c-42a9-8c0e-deeb21217c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:04.752092 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013dbe88-f97c-42a9-8c0e-deeb21217c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:04.752415 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013dbe88-f97c-42a9-8c0e-deeb21217c05 admin admin] PATH_INFO: `/v3/users/44f4d15284fa4a53906c2efca2d1f453` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:04.756540 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71a6a4-0cdd-4e19-9df3-18f01a60c9ad admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:04.756988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013dbe88-f97c-42a9-8c0e-deeb21217c05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44f4d15284fa4a53906c2efca2d1f453)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:04.764639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013dbe88-f97c-42a9-8c0e-deeb21217c05 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:04.771109 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17f4fd-fed8-4db8-a990-bd9a01fcad72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:04.774199 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17f4fd-fed8-4db8-a990-bd9a01fcad72 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:04.775008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17f4fd-fed8-4db8-a990-bd9a01fcad72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:04.775730 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17f4fd-fed8-4db8-a990-bd9a01fcad72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:04.777590 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d17f4fd-fed8-4db8-a990-bd9a01fcad72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:04.783097 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d17f4fd-fed8-4db8-a990-bd9a01fcad72 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:04.814538 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1323/2628] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:38:04.822059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17f4fd-fed8-4db8-a990-bd9a01fcad72 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:04.862484 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17f4fd-fed8-4db8-a990-bd9a01fcad72 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:04.887660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b309c76-9799-4862-9e01-55774fc0c868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:04.889552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b309c76-9799-4862-9e01-55774fc0c868 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:04.893932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b309c76-9799-4862-9e01-55774fc0c868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:04.894665 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b309c76-9799-4862-9e01-55774fc0c868 admin admin] PATH_INFO: `/v3/projects/17297f6951334b9f9ebd75da1e64eb0f/users/9db75b86ddc04f4e93e7bb8d9674d506/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:04.898489 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-013dbe88-f97c-42a9-8c0e-deeb21217c05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44f4d15284fa4a53906c2efca2d1f453'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:04.899982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b309c76-9799-4862-9e01-55774fc0c868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17297f6951334b9f9ebd75da1e64eb0f, user_id=9db75b86ddc04f4e93e7bb8d9674d506, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:04.904757 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b309c76-9799-4862-9e01-55774fc0c868 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:04.913862 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:04.923034 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1324/2629] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:38:04 2022] PUT /identity/v3/projects/17297f6951334b9f9ebd75da1e64eb0f/users/9db75b86ddc04f4e93e7bb8d9674d506/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:04.928523 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-013dbe88-f97c-42a9-8c0e-deeb21217c05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44f4d15284fa4a53906c2efca2d1f453'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:04.935664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39944be8-6909-4e07-a0d4-0a2f52b476f4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:04.935952 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39944be8-6909-4e07-a0d4-0a2f52b476f4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:04.936230 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39944be8-6909-4e07-a0d4-0a2f52b476f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:04.950191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-013dbe88-f97c-42a9-8c0e-deeb21217c05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44f4d15284fa4a53906c2efca2d1f453'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:04.956425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-013dbe88-f97c-42a9-8c0e-deeb21217c05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44f4d15284fa4a53906c2efca2d1f453'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:04.958126 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-013dbe88-f97c-42a9-8c0e-deeb21217c05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44f4d15284fa4a53906c2efca2d1f453'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:04.985610 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1306/2630] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:38:04 2022] DELETE /identity/v3/users/44f4d15284fa4a53906c2efca2d1f453 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:38:04.991564 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac642caa-e1f5-4e3e-91fb-6855ffd0ee0d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:04.992010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac642caa-e1f5-4e3e-91fb-6855ffd0ee0d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:04.992305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac642caa-e1f5-4e3e-91fb-6855ffd0ee0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:04.995957 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-39944be8-6909-4e07-a0d4-0a2f52b476f4 None None] MFA Rules not processed for user `9db75b86ddc04f4e93e7bb8d9674d506`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:04.998352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-39944be8-6909-4e07-a0d4-0a2f52b476f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:05.015128 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-ac642caa-e1f5-4e3e-91fb-6855ffd0ee0d None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:05.016428 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ac642caa-e1f5-4e3e-91fb-6855ffd0ee0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:05.118608 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1307/2631] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:38:05.123605 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1325/2632] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:05.133437 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac642caa-e1f5-4e3e-91fb-6855ffd0ee0d None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:05.137788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39944be8-6909-4e07-a0d4-0a2f52b476f4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:05.139278 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-39944be8-6909-4e07-a0d4-0a2f52b476f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:05.139688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ac642caa-e1f5-4e3e-91fb-6855ffd0ee0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:05.173950 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:05.220702 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39944be8-6909-4e07-a0d4-0a2f52b476f4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:05.225566 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d69186-d990-4f3a-b848-13042bec3ab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:05.226133 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac642caa-e1f5-4e3e-91fb-6855ffd0ee0d None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:05.226550 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d69186-d990-4f3a-b848-13042bec3ab4 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:05.227017 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d69186-d990-4f3a-b848-13042bec3ab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:05.227553 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d69186-d990-4f3a-b848-13042bec3ab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:05.232683 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41965e56-8bc8-426f-8a2f-27124c04dbb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:05.234053 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41965e56-8bc8-426f-8a2f-27124c04dbb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:05.234506 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41965e56-8bc8-426f-8a2f-27124c04dbb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:05.234947 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41965e56-8bc8-426f-8a2f-27124c04dbb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:05.248266 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d69186-d990-4f3a-b848-13042bec3ab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:05.250985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d69186-d990-4f3a-b848-13042bec3ab4 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:05.265810 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1326/2633] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:38:05.274924 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41965e56-8bc8-426f-8a2f-27124c04dbb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:05.277122 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41965e56-8bc8-426f-8a2f-27124c04dbb1 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:05.296832 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1308/2634] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:38:05.301492 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d69186-d990-4f3a-b848-13042bec3ab4 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:05.303049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d9d69186-d990-4f3a-b848-13042bec3ab4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:05.343280 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d69186-d990-4f3a-b848-13042bec3ab4 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:05.351705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c11471-d1e4-4bcb-94d1-e097632557b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:05.353895 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c11471-d1e4-4bcb-94d1-e097632557b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:05.355171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c11471-d1e4-4bcb-94d1-e097632557b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:05.384332 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c11471-d1e4-4bcb-94d1-e097632557b4 admin admin] PATH_INFO: `/v3/users/17e0316f7ea74eeeb675650cf6f6cfe6` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:05.387839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c11471-d1e4-4bcb-94d1-e097632557b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17e0316f7ea74eeeb675650cf6f6cfe6)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:05.391308 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c11471-d1e4-4bcb-94d1-e097632557b4 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:05.563395 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a1c11471-d1e4-4bcb-94d1-e097632557b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17e0316f7ea74eeeb675650cf6f6cfe6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:05.571792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a1c11471-d1e4-4bcb-94d1-e097632557b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17e0316f7ea74eeeb675650cf6f6cfe6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:05.598442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a1c11471-d1e4-4bcb-94d1-e097632557b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17e0316f7ea74eeeb675650cf6f6cfe6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:05.600053 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a1c11471-d1e4-4bcb-94d1-e097632557b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17e0316f7ea74eeeb675650cf6f6cfe6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:05.617537 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-a1c11471-d1e4-4bcb-94d1-e097632557b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17e0316f7ea74eeeb675650cf6f6cfe6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:05.653757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411dd5e-62b1-4f9a-9c90-625d93932269 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:05.654394 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411dd5e-62b1-4f9a-9c90-625d93932269 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:05.654615 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411dd5e-62b1-4f9a-9c90-625d93932269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:05.669401 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1327/2635] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:38:05 2022] DELETE /identity/v3/users/17e0316f7ea74eeeb675650cf6f6cfe6 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:05.679675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a411dd5e-62b1-4f9a-9c90-625d93932269 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:05.680962 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a411dd5e-62b1-4f9a-9c90-625d93932269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:05.703985 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c11471-d1e4-4bcb-94d1-e097632557b4 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:05.707907 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a1c11471-d1e4-4bcb-94d1-e097632557b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:05.837422 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c11471-d1e4-4bcb-94d1-e097632557b4 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:05.847082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ef777-b71d-4767-9d67-f12c1613f9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:05.848754 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ef777-b71d-4767-9d67-f12c1613f9dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:05.849412 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ef777-b71d-4767-9d67-f12c1613f9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:05.852610 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1309/2636] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:05.852711 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ef777-b71d-4767-9d67-f12c1613f9dc admin admin] PATH_INFO: `/v3/projects/dbf45826ee9545a982bd3e7a83a423fd` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:05.852711 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ef777-b71d-4767-9d67-f12c1613f9dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbf45826ee9545a982bd3e7a83a423fd)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:05.858253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ef777-b71d-4767-9d67-f12c1613f9dc admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:05.871069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411dd5e-62b1-4f9a-9c90-625d93932269 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:05.873452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a411dd5e-62b1-4f9a-9c90-625d93932269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:05.920831 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:05.962528 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:05.979478 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-161ef777-b71d-4767-9d67-f12c1613f9dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbf45826ee9545a982bd3e7a83a423fd', 'request_id': 'req-161ef777-b71d-4767-9d67-f12c1613f9dc'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:06.001340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-161ef777-b71d-4767-9d67-f12c1613f9dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbf45826ee9545a982bd3e7a83a423fd', 'request_id': 'req-161ef777-b71d-4767-9d67-f12c1613f9dc'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:06.014242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411dd5e-62b1-4f9a-9c90-625d93932269 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:06.022252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-161ef777-b71d-4767-9d67-f12c1613f9dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbf45826ee9545a982bd3e7a83a423fd', 'request_id': 'req-161ef777-b71d-4767-9d67-f12c1613f9dc'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:06.037419 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-161ef777-b71d-4767-9d67-f12c1613f9dc admin admin] The token cache is being invalidate because project dbf45826ee9545a982bd3e7a83a423fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:38:06.037719 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-161ef777-b71d-4767-9d67-f12c1613f9dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:06.038431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-161ef777-b71d-4767-9d67-f12c1613f9dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:06.041350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1328/2637] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:38:05 2022] DELETE /identity/v3/projects/dbf45826ee9545a982bd3e7a83a423fd => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:38:06.046438 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ef777-b71d-4767-9d67-f12c1613f9dc admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:06.048073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-161ef777-b71d-4767-9d67-f12c1613f9dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:06.142556 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ef777-b71d-4767-9d67-f12c1613f9dc admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:06.145879 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec45c4b-b715-4fb9-9365-105885d7dc54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:06.147502 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec45c4b-b715-4fb9-9365-105885d7dc54 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:06.148096 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec45c4b-b715-4fb9-9365-105885d7dc54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:06.148677 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec45c4b-b715-4fb9-9365-105885d7dc54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:06.149851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5bf25-7c3a-41eb-9d68-d4973c81275e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:06.151529 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5bf25-7c3a-41eb-9d68-d4973c81275e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:06.152658 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5bf25-7c3a-41eb-9d68-d4973c81275e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:06.152813 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5bf25-7c3a-41eb-9d68-d4973c81275e admin admin] PATH_INFO: `/v3/users/aac7a1fc1d0e4228a6c96f6bca0d2673` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:06.155316 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cec45c4b-b715-4fb9-9365-105885d7dc54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:06.158487 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d5bf25-7c3a-41eb-9d68-d4973c81275e admin admin] RBAC: Authorizing `identity:delete_user(user_id=aac7a1fc1d0e4228a6c96f6bca0d2673)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:06.160952 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d5bf25-7c3a-41eb-9d68-d4973c81275e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:06.181263 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec45c4b-b715-4fb9-9365-105885d7dc54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:06.184546 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec45c4b-b715-4fb9-9365-105885d7dc54 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:06.203910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1310/2638] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:38:06.256743 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f4d5bf25-7c3a-41eb-9d68-d4973c81275e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aac7a1fc1d0e4228a6c96f6bca0d2673'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:06.257568 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f4d5bf25-7c3a-41eb-9d68-d4973c81275e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aac7a1fc1d0e4228a6c96f6bca0d2673'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:06.288725 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f4d5bf25-7c3a-41eb-9d68-d4973c81275e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aac7a1fc1d0e4228a6c96f6bca0d2673'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:06.289808 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f4d5bf25-7c3a-41eb-9d68-d4973c81275e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aac7a1fc1d0e4228a6c96f6bca0d2673'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:06.313485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-f4d5bf25-7c3a-41eb-9d68-d4973c81275e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aac7a1fc1d0e4228a6c96f6bca0d2673'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:06.356360 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1329/2639] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:38:06 2022] DELETE /identity/v3/users/aac7a1fc1d0e4228a6c96f6bca0d2673 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:38:06.810288 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec45c4b-b715-4fb9-9365-105885d7dc54 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:06.812968 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-cec45c4b-b715-4fb9-9365-105885d7dc54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:06.851716 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec45c4b-b715-4fb9-9365-105885d7dc54 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:06.976762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608a6e4-7a60-4ef4-aa68-88018b350cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:06.978099 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608a6e4-7a60-4ef4-aa68-88018b350cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:06.978620 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608a6e4-7a60-4ef4-aa68-88018b350cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:06.979175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608a6e4-7a60-4ef4-aa68-88018b350cab admin admin] PATH_INFO: `/v3/projects/198b9677b0a04e0ead76704f5c5b1aba` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:06.982216 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8608a6e4-7a60-4ef4-aa68-88018b350cab admin admin] RBAC: Authorizing `identity:delete_project(project_id=198b9677b0a04e0ead76704f5c5b1aba)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:06.985318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8608a6e4-7a60-4ef4-aa68-88018b350cab admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:07.052653 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5bf25-7c3a-41eb-9d68-d4973c81275e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:07.054387 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f4d5bf25-7c3a-41eb-9d68-d4973c81275e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:07.094453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5bf25-7c3a-41eb-9d68-d4973c81275e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:07.114560 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8608a6e4-7a60-4ef4-aa68-88018b350cab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '198b9677b0a04e0ead76704f5c5b1aba', 'request_id': 'req-8608a6e4-7a60-4ef4-aa68-88018b350cab'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:07.133454 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8608a6e4-7a60-4ef4-aa68-88018b350cab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '198b9677b0a04e0ead76704f5c5b1aba', 'request_id': 'req-8608a6e4-7a60-4ef4-aa68-88018b350cab'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:07.143400 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8608a6e4-7a60-4ef4-aa68-88018b350cab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '198b9677b0a04e0ead76704f5c5b1aba', 'request_id': 'req-8608a6e4-7a60-4ef4-aa68-88018b350cab'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:07.159888 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8608a6e4-7a60-4ef4-aa68-88018b350cab admin admin] The token cache is being invalidate because project 198b9677b0a04e0ead76704f5c5b1aba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:38:07.160580 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8608a6e4-7a60-4ef4-aa68-88018b350cab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:07.161704 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-8608a6e4-7a60-4ef4-aa68-88018b350cab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:07.170170 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1311/2640] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:38:06 2022] DELETE /identity/v3/projects/198b9677b0a04e0ead76704f5c5b1aba => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:07.171123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608a6e4-7a60-4ef4-aa68-88018b350cab admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:07.172959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8608a6e4-7a60-4ef4-aa68-88018b350cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:07.193836 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:07.230090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:07.253162 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95198854-e868-4755-9d7e-e9918ee1ba0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:07.254422 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95198854-e868-4755-9d7e-e9918ee1ba0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:07.254876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95198854-e868-4755-9d7e-e9918ee1ba0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:07.256555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95198854-e868-4755-9d7e-e9918ee1ba0f admin admin] PATH_INFO: `/v3/projects/4ea15701a40a4a789cdf72038a5e2456` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:07.258386 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95198854-e868-4755-9d7e-e9918ee1ba0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ea15701a40a4a789cdf72038a5e2456)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:07.261247 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95198854-e868-4755-9d7e-e9918ee1ba0f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:07.327016 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608a6e4-7a60-4ef4-aa68-88018b350cab admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:07.344655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a675b-0b70-4ef8-bd72-4112c80ecf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:07.347387 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a675b-0b70-4ef8-bd72-4112c80ecf60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:07.348464 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a675b-0b70-4ef8-bd72-4112c80ecf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:07.350118 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a675b-0b70-4ef8-bd72-4112c80ecf60 admin admin] PATH_INFO: `/v3/users/9db75b86ddc04f4e93e7bb8d9674d506` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:07.352761 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a675b-0b70-4ef8-bd72-4112c80ecf60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9db75b86ddc04f4e93e7bb8d9674d506)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:07.355956 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a675b-0b70-4ef8-bd72-4112c80ecf60 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:07.381968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-95198854-e868-4755-9d7e-e9918ee1ba0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ea15701a40a4a789cdf72038a5e2456', 'request_id': 'req-95198854-e868-4755-9d7e-e9918ee1ba0f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:07.395117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-95198854-e868-4755-9d7e-e9918ee1ba0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ea15701a40a4a789cdf72038a5e2456', 'request_id': 'req-95198854-e868-4755-9d7e-e9918ee1ba0f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:07.411377 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-95198854-e868-4755-9d7e-e9918ee1ba0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ea15701a40a4a789cdf72038a5e2456', 'request_id': 'req-95198854-e868-4755-9d7e-e9918ee1ba0f'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:07.441170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-95198854-e868-4755-9d7e-e9918ee1ba0f admin admin] The token cache is being invalidate because project 4ea15701a40a4a789cdf72038a5e2456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:38:07.441710 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-95198854-e868-4755-9d7e-e9918ee1ba0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:07.443111 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-95198854-e868-4755-9d7e-e9918ee1ba0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:07.446857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1330/2641] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:38:07 2022] DELETE /identity/v3/projects/4ea15701a40a4a789cdf72038a5e2456 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:38:07.459781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85781d-eb9d-4c27-8fac-e64a37651f66 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:07.461144 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85781d-eb9d-4c27-8fac-e64a37651f66 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:07.462073 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85781d-eb9d-4c27-8fac-e64a37651f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:07.462436 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-051a675b-0b70-4ef8-bd72-4112c80ecf60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9db75b86ddc04f4e93e7bb8d9674d506'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:07.495548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-051a675b-0b70-4ef8-bd72-4112c80ecf60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9db75b86ddc04f4e93e7bb8d9674d506'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:07.508211 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-2e85781d-eb9d-4c27-8fac-e64a37651f66 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:07.513785 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-051a675b-0b70-4ef8-bd72-4112c80ecf60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9db75b86ddc04f4e93e7bb8d9674d506'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:07.516029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2e85781d-eb9d-4c27-8fac-e64a37651f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:07.516856 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-051a675b-0b70-4ef8-bd72-4112c80ecf60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9db75b86ddc04f4e93e7bb8d9674d506'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:07.518147 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-051a675b-0b70-4ef8-bd72-4112c80ecf60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9db75b86ddc04f4e93e7bb8d9674d506'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:07.551225 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1312/2642] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:38:07 2022] DELETE /identity/v3/users/9db75b86ddc04f4e93e7bb8d9674d506 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:07.622229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1331/2643] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:07.643900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d18651-25c5-430e-8f60-44e8b1696aa0 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:07.644440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d18651-25c5-430e-8f60-44e8b1696aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:07.644970 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d18651-25c5-430e-8f60-44e8b1696aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:07.723520 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b5d18651-25c5-430e-8f60-44e8b1696aa0 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:07.730128 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b5d18651-25c5-430e-8f60-44e8b1696aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:07.927450 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1313/2644] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:07.939930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85781d-eb9d-4c27-8fac-e64a37651f66 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:07.961248 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85781d-eb9d-4c27-8fac-e64a37651f66 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:07.973042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85733d52-7459-4bef-9849-16813f8871a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:07.975035 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85733d52-7459-4bef-9849-16813f8871a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:07.975035 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85733d52-7459-4bef-9849-16813f8871a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:07.975628 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85733d52-7459-4bef-9849-16813f8871a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:07.977716 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85733d52-7459-4bef-9849-16813f8871a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:38:07.977991 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85733d52-7459-4bef-9849-16813f8871a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:07.983546 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85733d52-7459-4bef-9849-16813f8871a1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:08.025358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1332/2645] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:38:08.030392 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d18651-25c5-430e-8f60-44e8b1696aa0 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:08.069895 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d18651-25c5-430e-8f60-44e8b1696aa0 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:08.092304 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80cfd8-b256-4f7a-8cb3-3720c6d2cdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:08.093118 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80cfd8-b256-4f7a-8cb3-3720c6d2cdd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:08.094136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80cfd8-b256-4f7a-8cb3-3720c6d2cdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:08.094136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80cfd8-b256-4f7a-8cb3-3720c6d2cdd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:08.094733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80cfd8-b256-4f7a-8cb3-3720c6d2cdd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:08.096859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80cfd8-b256-4f7a-8cb3-3720c6d2cdd6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:08.156649 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1314/2646] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:38:08 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:38:08.164926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85733d52-7459-4bef-9849-16813f8871a1 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:08.188816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85733d52-7459-4bef-9849-16813f8871a1 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:08.202904 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80cfd8-b256-4f7a-8cb3-3720c6d2cdd6 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:08.204172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fa80cfd8-b256-4f7a-8cb3-3720c6d2cdd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:08.209766 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:08.394139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80cfd8-b256-4f7a-8cb3-3720c6d2cdd6 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:08.405246 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116df41d-0392-40c6-875c-10087f6b92b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:08.407230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116df41d-0392-40c6-875c-10087f6b92b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:08.407994 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116df41d-0392-40c6-875c-10087f6b92b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:08.408760 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116df41d-0392-40c6-875c-10087f6b92b8 admin admin] PATH_INFO: `/v3/projects/17297f6951334b9f9ebd75da1e64eb0f` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:08.413425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116df41d-0392-40c6-875c-10087f6b92b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17297f6951334b9f9ebd75da1e64eb0f)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:08.418911 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116df41d-0392-40c6-875c-10087f6b92b8 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:08.423184 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bdf70-9f18-476c-9f78-499c40083383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:08.425565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bdf70-9f18-476c-9f78-499c40083383 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:08.432201 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bdf70-9f18-476c-9f78-499c40083383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:08.433423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bdf70-9f18-476c-9f78-499c40083383 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:08.435607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5bdf70-9f18-476c-9f78-499c40083383 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:08.439687 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5bdf70-9f18-476c-9f78-499c40083383 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:08.462100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:08.500124 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1333/2647] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:38:08 2022] POST /identity/v3/users => generated 293 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:38:08.502391 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bdf70-9f18-476c-9f78-499c40083383 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:08.522845 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bdf70-9f18-476c-9f78-499c40083383 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:08.525735 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-116df41d-0392-40c6-875c-10087f6b92b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17297f6951334b9f9ebd75da1e64eb0f', 'request_id': 'req-116df41d-0392-40c6-875c-10087f6b92b8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:08.539979 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-116df41d-0392-40c6-875c-10087f6b92b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17297f6951334b9f9ebd75da1e64eb0f', 'request_id': 'req-116df41d-0392-40c6-875c-10087f6b92b8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:08.566693 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-116df41d-0392-40c6-875c-10087f6b92b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17297f6951334b9f9ebd75da1e64eb0f', 'request_id': 'req-116df41d-0392-40c6-875c-10087f6b92b8'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:08.594646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-116df41d-0392-40c6-875c-10087f6b92b8 admin admin] The token cache is being invalidate because project 17297f6951334b9f9ebd75da1e64eb0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:38:08.595260 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-116df41d-0392-40c6-875c-10087f6b92b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:08.596545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-116df41d-0392-40c6-875c-10087f6b92b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:08.599681 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1315/2648] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:38:08 2022] DELETE /identity/v3/projects/17297f6951334b9f9ebd75da1e64eb0f => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:38:08.606584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54208d-918e-4e8e-b9d4-ac8d20f5b993 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:08.607219 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54208d-918e-4e8e-b9d4-ac8d20f5b993 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:08.607829 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54208d-918e-4e8e-b9d4-ac8d20f5b993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:08.648736 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-bf54208d-918e-4e8e-b9d4-ac8d20f5b993 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:08.648951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c80940-dd40-4add-aa35-ed60efd6f356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:08.648951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c80940-dd40-4add-aa35-ed60efd6f356 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:08.648951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c80940-dd40-4add-aa35-ed60efd6f356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:08.648951 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c80940-dd40-4add-aa35-ed60efd6f356 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:08.649965 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c80940-dd40-4add-aa35-ed60efd6f356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:08.650247 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bf54208d-918e-4e8e-b9d4-ac8d20f5b993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:08.653524 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c80940-dd40-4add-aa35-ed60efd6f356 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:08.670202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1334/2649] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:38:08.672402 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1316/2650] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:38:08.674170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c80940-dd40-4add-aa35-ed60efd6f356 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:08.677190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-68c80940-dd40-4add-aa35-ed60efd6f356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:08.682667 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a46e24-7bec-42b1-8208-45a0810bd251 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:08.683919 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a46e24-7bec-42b1-8208-45a0810bd251 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:08.684516 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a46e24-7bec-42b1-8208-45a0810bd251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:08.700962 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c80940-dd40-4add-aa35-ed60efd6f356 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:08.707954 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9d12f-b871-4325-b31f-3be0613e5df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:08.709087 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9d12f-b871-4325-b31f-3be0613e5df1 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:08.709633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9d12f-b871-4325-b31f-3be0613e5df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:08.710143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9d12f-b871-4325-b31f-3be0613e5df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:08.711259 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9d12f-b871-4325-b31f-3be0613e5df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:08.717591 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9d12f-b871-4325-b31f-3be0613e5df1 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:08.730191 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-14a46e24-7bec-42b1-8208-45a0810bd251 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:08.732155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1335/2651] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:38:08.733271 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-14a46e24-7bec-42b1-8208-45a0810bd251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:08.736094 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9d12f-b871-4325-b31f-3be0613e5df1 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:08.748440 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9d12f-b871-4325-b31f-3be0613e5df1 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:08.757014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1317/2652] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:08.759249 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a46e24-7bec-42b1-8208-45a0810bd251 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:08.760506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51dae2-6dbd-4a77-b3fe-5ac8c61e74ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:08.761740 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51dae2-6dbd-4a77-b3fe-5ac8c61e74ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:08.762232 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51dae2-6dbd-4a77-b3fe-5ac8c61e74ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:08.762715 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51dae2-6dbd-4a77-b3fe-5ac8c61e74ea admin admin] PATH_INFO: `/v3/projects/631e7f55ea434c3f9ddbb9ec32e232c3/users/87467340681a4c09aeae67a22f2494fe/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:08.781271 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a46e24-7bec-42b1-8208-45a0810bd251 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:08.784168 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51dae2-6dbd-4a77-b3fe-5ac8c61e74ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=631e7f55ea434c3f9ddbb9ec32e232c3, user_id=87467340681a4c09aeae67a22f2494fe, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:08.788318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51dae2-6dbd-4a77-b3fe-5ac8c61e74ea admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:08.793479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28a20d-0634-4319-9efc-4bc5193f2f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:08.794785 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28a20d-0634-4319-9efc-4bc5193f2f56 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:08.795544 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28a20d-0634-4319-9efc-4bc5193f2f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:08.796206 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28a20d-0634-4319-9efc-4bc5193f2f56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:08.799184 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28a20d-0634-4319-9efc-4bc5193f2f56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:38:08.799609 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28a20d-0634-4319-9efc-4bc5193f2f56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:08.802900 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28a20d-0634-4319-9efc-4bc5193f2f56 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:08.813346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1336/2653] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:38:08 2022] PUT /identity/v3/projects/631e7f55ea434c3f9ddbb9ec32e232c3/users/87467340681a4c09aeae67a22f2494fe/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:08.820940 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1318/2654] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:38:08.835307 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d48fe1-8eac-49c4-8db5-06607f39b883 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:08.836377 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51dae2-6dbd-4a77-b3fe-5ac8c61e74ea admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:08.837020 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d48fe1-8eac-49c4-8db5-06607f39b883 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:08.837937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d48fe1-8eac-49c4-8db5-06607f39b883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:08.894393 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-26d48fe1-8eac-49c4-8db5-06607f39b883 None None] MFA Rules not processed for user `87467340681a4c09aeae67a22f2494fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:08.895675 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-26d48fe1-8eac-49c4-8db5-06607f39b883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:08.978193 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51dae2-6dbd-4a77-b3fe-5ac8c61e74ea admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:08.994941 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1488513-5fa0-4615-af5c-2d8dcb854c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:08.996791 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1488513-5fa0-4615-af5c-2d8dcb854c34 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:08.997469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1488513-5fa0-4615-af5c-2d8dcb854c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:08.998059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1488513-5fa0-4615-af5c-2d8dcb854c34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:08.999541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1488513-5fa0-4615-af5c-2d8dcb854c34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:09.002840 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1488513-5fa0-4615-af5c-2d8dcb854c34 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:09.013549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:09.043276 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1319/2655] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:09.047441 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1337/2656] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:38:08 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:38:09.051771 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d48fe1-8eac-49c4-8db5-06607f39b883 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:09.054416 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1488513-5fa0-4615-af5c-2d8dcb854c34 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:09.058704 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f1488513-5fa0-4615-af5c-2d8dcb854c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:09.064662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d48fe1-8eac-49c4-8db5-06607f39b883 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:09.159401 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30884789-836d-44fb-b988-52ab2d27a7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:09.160593 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30884789-836d-44fb-b988-52ab2d27a7ca admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:09.161141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30884789-836d-44fb-b988-52ab2d27a7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:09.161755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30884789-836d-44fb-b988-52ab2d27a7ca admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:09.162901 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30884789-836d-44fb-b988-52ab2d27a7ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:09.165372 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30884789-836d-44fb-b988-52ab2d27a7ca admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:09.170818 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:09.198549 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1488513-5fa0-4615-af5c-2d8dcb854c34 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:09.204252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c4086-5598-478a-8dd8-bfdef889b623 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:09.205047 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c4086-5598-478a-8dd8-bfdef889b623 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:09.205444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1320/2657] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:38:09 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:38:09.205581 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c4086-5598-478a-8dd8-bfdef889b623 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:09.205852 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c4086-5598-478a-8dd8-bfdef889b623 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:09.212083 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30884789-836d-44fb-b988-52ab2d27a7ca admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:09.231772 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30884789-836d-44fb-b988-52ab2d27a7ca admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:09.241161 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48bbf7-66b0-4c55-acf5-f99bbb543f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:09.242350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48bbf7-66b0-4c55-acf5-f99bbb543f34 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:09.242848 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48bbf7-66b0-4c55-acf5-f99bbb543f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:09.243339 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48bbf7-66b0-4c55-acf5-f99bbb543f34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:09.245492 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48bbf7-66b0-4c55-acf5-f99bbb543f34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:09.248753 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48bbf7-66b0-4c55-acf5-f99bbb543f34 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:09.269402 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1321/2658] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:38:09.274512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48bbf7-66b0-4c55-acf5-f99bbb543f34 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:09.292608 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48bbf7-66b0-4c55-acf5-f99bbb543f34 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:09.300693 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4fec3-0317-4269-843e-013f08180443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:09.302881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4fec3-0317-4269-843e-013f08180443 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:09.302881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4fec3-0317-4269-843e-013f08180443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:09.302881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4fec3-0317-4269-843e-013f08180443 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:09.305913 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4fec3-0317-4269-843e-013f08180443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:09.305913 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4fec3-0317-4269-843e-013f08180443 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:09.310989 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c4086-5598-478a-8dd8-bfdef889b623 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:09.312942 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c4086-5598-478a-8dd8-bfdef889b623 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:09.337713 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1322/2659] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:38:09.341422 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1338/2660] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:38:09.343105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4fec3-0317-4269-843e-013f08180443 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:09.362766 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4fec3-0317-4269-843e-013f08180443 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:09.373595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fa72b-dcf7-451b-869b-1fc1d8b6ab57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:09.374749 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fa72b-dcf7-451b-869b-1fc1d8b6ab57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:09.375414 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fa72b-dcf7-451b-869b-1fc1d8b6ab57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:09.376410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fa72b-dcf7-451b-869b-1fc1d8b6ab57 admin admin] PATH_INFO: `/v3/projects/3f9653e7c11b43fab416ed3530a4b65d/users/8c66f85c56454f4fa91d37f2b6193a58/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:09.428300 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48fa72b-dcf7-451b-869b-1fc1d8b6ab57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f9653e7c11b43fab416ed3530a4b65d, user_id=8c66f85c56454f4fa91d37f2b6193a58, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:09.431542 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48fa72b-dcf7-451b-869b-1fc1d8b6ab57 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:09.449344 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1323/2661] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:38:09 2022] PUT /identity/v3/projects/3f9653e7c11b43fab416ed3530a4b65d/users/8c66f85c56454f4fa91d37f2b6193a58/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:38:09.460873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374eab3-199b-4f35-b344-2801a580f1f4 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:09.461213 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374eab3-199b-4f35-b344-2801a580f1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:09.461509 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374eab3-199b-4f35-b344-2801a580f1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:09.502349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-1374eab3-199b-4f35-b344-2801a580f1f4 None None] MFA Rules not processed for user `8c66f85c56454f4fa91d37f2b6193a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:09.504186 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1374eab3-199b-4f35-b344-2801a580f1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:09.624969 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1339/2662] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:38:09.631957 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fa72b-dcf7-451b-869b-1fc1d8b6ab57 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:09.638105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f48fa72b-dcf7-451b-869b-1fc1d8b6ab57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:09.748111 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fa72b-dcf7-451b-869b-1fc1d8b6ab57 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:09.760799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354e3db-4e8e-42ec-b1c1-60a910278e45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:09.761770 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354e3db-4e8e-42ec-b1c1-60a910278e45 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:09.762189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354e3db-4e8e-42ec-b1c1-60a910278e45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:09.762479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354e3db-4e8e-42ec-b1c1-60a910278e45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:09.788300 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354e3db-4e8e-42ec-b1c1-60a910278e45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:09.790280 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354e3db-4e8e-42ec-b1c1-60a910278e45 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:09.819125 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1324/2663] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:38:09.863952 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374eab3-199b-4f35-b344-2801a580f1f4 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:10.007962 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374eab3-199b-4f35-b344-2801a580f1f4 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:10.015150 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c03d0b-df65-4ca1-ab95-82ac7a454dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:10.016926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c03d0b-df65-4ca1-ab95-82ac7a454dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:10.017210 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c03d0b-df65-4ca1-ab95-82ac7a454dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:10.017530 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c03d0b-df65-4ca1-ab95-82ac7a454dd8 admin admin] PATH_INFO: `/v3/users/8c66f85c56454f4fa91d37f2b6193a58` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:10.019249 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c03d0b-df65-4ca1-ab95-82ac7a454dd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c66f85c56454f4fa91d37f2b6193a58)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:10.021506 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c03d0b-df65-4ca1-ab95-82ac7a454dd8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:10.089756 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:10.123909 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-23c03d0b-df65-4ca1-ab95-82ac7a454dd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c66f85c56454f4fa91d37f2b6193a58'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:10.124639 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-23c03d0b-df65-4ca1-ab95-82ac7a454dd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c66f85c56454f4fa91d37f2b6193a58'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:10.143243 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-23c03d0b-df65-4ca1-ab95-82ac7a454dd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c66f85c56454f4fa91d37f2b6193a58'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:10.144027 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-23c03d0b-df65-4ca1-ab95-82ac7a454dd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c66f85c56454f4fa91d37f2b6193a58'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:10.156240 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-23c03d0b-df65-4ca1-ab95-82ac7a454dd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c66f85c56454f4fa91d37f2b6193a58'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:10.187284 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1340/2664] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:38:09 2022] DELETE /identity/v3/users/8c66f85c56454f4fa91d37f2b6193a58 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:10.196253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d618b59-670a-4f3b-a587-50a45e14b081 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:10.197063 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d618b59-670a-4f3b-a587-50a45e14b081 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:10.197854 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d618b59-670a-4f3b-a587-50a45e14b081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:10.231984 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1d618b59-670a-4f3b-a587-50a45e14b081 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:10.233668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1d618b59-670a-4f3b-a587-50a45e14b081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:10.301014 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:10.340360 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1325/2665] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:10.356971 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c03d0b-df65-4ca1-ab95-82ac7a454dd8 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:10.358955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-23c03d0b-df65-4ca1-ab95-82ac7a454dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:10.477876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c03d0b-df65-4ca1-ab95-82ac7a454dd8 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:10.484350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2c534-cf0d-400e-8a58-e464ef06a9d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:10.485311 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2c534-cf0d-400e-8a58-e464ef06a9d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:10.485738 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2c534-cf0d-400e-8a58-e464ef06a9d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:10.486147 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2c534-cf0d-400e-8a58-e464ef06a9d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:10.499082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f2c534-cf0d-400e-8a58-e464ef06a9d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:10.501675 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f2c534-cf0d-400e-8a58-e464ef06a9d9 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:10.521561 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1341/2666] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:38:10.941910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d618b59-670a-4f3b-a587-50a45e14b081 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:10.944141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1d618b59-670a-4f3b-a587-50a45e14b081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:10.965986 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d618b59-670a-4f3b-a587-50a45e14b081 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:10.974013 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d214f-c5b2-4afb-88ca-67d753c98244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:10.975134 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d214f-c5b2-4afb-88ca-67d753c98244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:10.975553 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d214f-c5b2-4afb-88ca-67d753c98244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:10.975979 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d214f-c5b2-4afb-88ca-67d753c98244 admin admin] PATH_INFO: `/v3/projects/3f9653e7c11b43fab416ed3530a4b65d` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:10.977845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d214f-c5b2-4afb-88ca-67d753c98244 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f9653e7c11b43fab416ed3530a4b65d)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:10.981699 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d214f-c5b2-4afb-88ca-67d753c98244 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:11.082306 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-060d214f-c5b2-4afb-88ca-67d753c98244 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f9653e7c11b43fab416ed3530a4b65d', 'request_id': 'req-060d214f-c5b2-4afb-88ca-67d753c98244'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:11.099269 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-060d214f-c5b2-4afb-88ca-67d753c98244 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f9653e7c11b43fab416ed3530a4b65d', 'request_id': 'req-060d214f-c5b2-4afb-88ca-67d753c98244'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:11.115909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-060d214f-c5b2-4afb-88ca-67d753c98244 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f9653e7c11b43fab416ed3530a4b65d', 'request_id': 'req-060d214f-c5b2-4afb-88ca-67d753c98244'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:11.138624 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-060d214f-c5b2-4afb-88ca-67d753c98244 admin admin] The token cache is being invalidate because project 3f9653e7c11b43fab416ed3530a4b65d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:38:11.140018 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-060d214f-c5b2-4afb-88ca-67d753c98244 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:11.140246 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-060d214f-c5b2-4afb-88ca-67d753c98244 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:11.142770 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1326/2667] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:38:10 2022] DELETE /identity/v3/projects/3f9653e7c11b43fab416ed3530a4b65d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:11.149202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8c4c0-b1fc-4e1e-9d31-52c3945c2425 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:11.149560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8c4c0-b1fc-4e1e-9d31-52c3945c2425 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:11.149806 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8c4c0-b1fc-4e1e-9d31-52c3945c2425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:11.176775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-6dd8c4c0-b1fc-4e1e-9d31-52c3945c2425 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:11.178443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6dd8c4c0-b1fc-4e1e-9d31-52c3945c2425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:11.306735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1342/2668] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:38:11.316093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b4f21-80c5-48eb-950a-41f1080711f9 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:11.317413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b4f21-80c5-48eb-950a-41f1080711f9 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:11.318160 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b4f21-80c5-48eb-950a-41f1080711f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:11.348704 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-a35b4f21-80c5-48eb-950a-41f1080711f9 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:11.350957 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a35b4f21-80c5-48eb-950a-41f1080711f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:11.361728 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1327/2669] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:11.366323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8c4c0-b1fc-4e1e-9d31-52c3945c2425 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:11.377597 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8c4c0-b1fc-4e1e-9d31-52c3945c2425 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:11.382622 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10514b9a-b176-4eba-ac0d-6b903f7c44ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:11.383401 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10514b9a-b176-4eba-ac0d-6b903f7c44ed admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:11.383757 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10514b9a-b176-4eba-ac0d-6b903f7c44ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:11.384070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10514b9a-b176-4eba-ac0d-6b903f7c44ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:11.384834 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10514b9a-b176-4eba-ac0d-6b903f7c44ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:38:11.386082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10514b9a-b176-4eba-ac0d-6b903f7c44ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:11.387789 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10514b9a-b176-4eba-ac0d-6b903f7c44ed admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:11.401895 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1343/2670] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:38:11.407238 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b4f21-80c5-48eb-950a-41f1080711f9 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:11.420872 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b4f21-80c5-48eb-950a-41f1080711f9 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:11.427018 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ef839-f8f0-43f0-a1aa-3b702a5f1bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:11.427688 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ef839-f8f0-43f0-a1aa-3b702a5f1bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:11.428097 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ef839-f8f0-43f0-a1aa-3b702a5f1bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:11.428245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ef839-f8f0-43f0-a1aa-3b702a5f1bb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:11.428947 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ef839-f8f0-43f0-a1aa-3b702a5f1bb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:11.431131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ef839-f8f0-43f0-a1aa-3b702a5f1bb7 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:11.464283 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1328/2671] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:38:11 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:38:11.470039 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10514b9a-b176-4eba-ac0d-6b903f7c44ed admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:11.485450 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10514b9a-b176-4eba-ac0d-6b903f7c44ed admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:11.531853 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:11.567465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca957c-0a64-436d-b774-d1f0d09456a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:11.568166 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca957c-0a64-436d-b774-d1f0d09456a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:11.568450 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca957c-0a64-436d-b774-d1f0d09456a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:11.568746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca957c-0a64-436d-b774-d1f0d09456a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:11.569505 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca957c-0a64-436d-b774-d1f0d09456a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:11.571476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca957c-0a64-436d-b774-d1f0d09456a6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:11.593697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1344/2672] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:38:11 2022] POST /identity/v3/users => generated 290 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:38:11.597961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ef839-f8f0-43f0-a1aa-3b702a5f1bb7 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:11.610105 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ef839-f8f0-43f0-a1aa-3b702a5f1bb7 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:11.615103 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b2760-5e66-435a-afd1-864281414475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:11.615795 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b2760-5e66-435a-afd1-864281414475 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:11.616084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b2760-5e66-435a-afd1-864281414475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:11.616367 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b2760-5e66-435a-afd1-864281414475 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:11.617103 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b2760-5e66-435a-afd1-864281414475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:11.618812 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b2760-5e66-435a-afd1-864281414475 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:11.632641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1329/2673] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:38:11.637734 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca957c-0a64-436d-b774-d1f0d09456a6 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:11.648496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca957c-0a64-436d-b774-d1f0d09456a6 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:11.654090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b092a-945b-41fb-bf03-71fc5247a639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:11.654932 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b092a-945b-41fb-bf03-71fc5247a639 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:11.655222 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b092a-945b-41fb-bf03-71fc5247a639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:11.655515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b092a-945b-41fb-bf03-71fc5247a639 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:11.656281 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b092a-945b-41fb-bf03-71fc5247a639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:11.658018 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b092a-945b-41fb-bf03-71fc5247a639 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:11.670470 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1345/2674] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:38:11.676520 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b2760-5e66-435a-afd1-864281414475 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:11.688990 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b2760-5e66-435a-afd1-864281414475 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:11.694345 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe653e-d3d7-4e18-b7b6-43a9cfac2e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:11.695252 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe653e-d3d7-4e18-b7b6-43a9cfac2e38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:11.695553 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe653e-d3d7-4e18-b7b6-43a9cfac2e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:11.695830 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe653e-d3d7-4e18-b7b6-43a9cfac2e38 admin admin] PATH_INFO: `/v3/projects/318e4e0d9f3a49f69eb3d846539b6b67/users/c1815cc5a2fd43259c5b53650dbc7603/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:11.716819 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe653e-d3d7-4e18-b7b6-43a9cfac2e38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=318e4e0d9f3a49f69eb3d846539b6b67, user_id=c1815cc5a2fd43259c5b53650dbc7603, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:11.722573 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe653e-d3d7-4e18-b7b6-43a9cfac2e38 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:11.742586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1330/2675] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:38:11 2022] PUT /identity/v3/projects/318e4e0d9f3a49f69eb3d846539b6b67/users/c1815cc5a2fd43259c5b53650dbc7603/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:11.749019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fe5c7-135c-4665-8fbd-5a0c80cef424 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:11.749418 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fe5c7-135c-4665-8fbd-5a0c80cef424 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:11.749762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fe5c7-135c-4665-8fbd-5a0c80cef424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:11.771104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-4b6fe5c7-135c-4665-8fbd-5a0c80cef424 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:11.772350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4b6fe5c7-135c-4665-8fbd-5a0c80cef424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:11.843461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1346/2676] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:11.854507 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe653e-d3d7-4e18-b7b6-43a9cfac2e38 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:11.856347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-67fe653e-d3d7-4e18-b7b6-43a9cfac2e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:11.924587 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe653e-d3d7-4e18-b7b6-43a9cfac2e38 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:11.928445 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0ca08-724e-4417-a3e7-65a83035a356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:11.929115 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0ca08-724e-4417-a3e7-65a83035a356 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:11.929450 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0ca08-724e-4417-a3e7-65a83035a356 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:11.929745 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0ca08-724e-4417-a3e7-65a83035a356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:11.942221 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0ca08-724e-4417-a3e7-65a83035a356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:11.943835 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0ca08-724e-4417-a3e7-65a83035a356 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:11.955141 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1331/2677] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:38:17.548485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e1ba9-ffe6-442b-a5af-6f9126e826fd None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:17.549224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e1ba9-ffe6-442b-a5af-6f9126e826fd None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:17.550328 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e1ba9-ffe6-442b-a5af-6f9126e826fd None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:17.552348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1347/2678] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 06:38:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:38:17.564665 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2492b7-2181-4969-a74c-2bdf2951c6ad None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:17.565612 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2492b7-2181-4969-a74c-2bdf2951c6ad None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:17.566053 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2492b7-2181-4969-a74c-2bdf2951c6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:17.576720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:17.638275 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-7b2492b7-2181-4969-a74c-2bdf2951c6ad None None] MFA Rules not processed for user `2a09f4878eac4d60a9a7dcedba15c05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:17.642591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7b2492b7-2181-4969-a74c-2bdf2951c6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:17.831778 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1332/2679] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 06:38:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:38:17.844220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e1ba9-ffe6-442b-a5af-6f9126e826fd None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:17.846992 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-304e1ba9-ffe6-442b-a5af-6f9126e826fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:17.847379 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2492b7-2181-4969-a74c-2bdf2951c6ad None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:17.848975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7b2492b7-2181-4969-a74c-2bdf2951c6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:17.893195 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e1ba9-ffe6-442b-a5af-6f9126e826fd None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:17.904342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17973ba4-7787-4e0f-a288-a29fda4b61d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:17.905318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17973ba4-7787-4e0f-a288-a29fda4b61d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:17.905611 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17973ba4-7787-4e0f-a288-a29fda4b61d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:17.905891 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17973ba4-7787-4e0f-a288-a29fda4b61d7 admin admin] PATH_INFO: `/v3/users/87467340681a4c09aeae67a22f2494fe` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:17.907875 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17973ba4-7787-4e0f-a288-a29fda4b61d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87467340681a4c09aeae67a22f2494fe)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:17.910885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17973ba4-7787-4e0f-a288-a29fda4b61d7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:17.981138 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2492b7-2181-4969-a74c-2bdf2951c6ad None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:17.993068 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d972e-364c-4362-a3de-4814983f39b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a812c3b9b20f44f8814051381a8005b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:17.996785 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d972e-364c-4362-a3de-4814983f39b3 service glance] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:17.997190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d972e-364c-4362-a3de-4814983f39b3 service glance] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:17.997512 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d972e-364c-4362-a3de-4814983f39b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:18.024973 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-17973ba4-7787-4e0f-a288-a29fda4b61d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87467340681a4c09aeae67a22f2494fe'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:18.026897 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-17973ba4-7787-4e0f-a288-a29fda4b61d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87467340681a4c09aeae67a22f2494fe'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:18.029160 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d972e-364c-4362-a3de-4814983f39b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:18.030355 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d972e-364c-4362-a3de-4814983f39b3 service glance] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:18.061881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-17973ba4-7787-4e0f-a288-a29fda4b61d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87467340681a4c09aeae67a22f2494fe'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:18.063213 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-17973ba4-7787-4e0f-a288-a29fda4b61d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87467340681a4c09aeae67a22f2494fe'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:18.075631 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-17973ba4-7787-4e0f-a288-a29fda4b61d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87467340681a4c09aeae67a22f2494fe'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:18.082534 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1333/2680] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:38:18.106185 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1348/2681] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:38:17 2022] DELETE /identity/v3/users/87467340681a4c09aeae67a22f2494fe => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:18.110905 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce1d8f-cc1a-4fc9-9eb3-a970dd3ac521 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:18.111480 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce1d8f-cc1a-4fc9-9eb3-a970dd3ac521 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:18.114508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce1d8f-cc1a-4fc9-9eb3-a970dd3ac521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:18.141483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-42ce1d8f-cc1a-4fc9-9eb3-a970dd3ac521 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:18.145044 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-42ce1d8f-cc1a-4fc9-9eb3-a970dd3ac521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:18.210361 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a870c8-e1bd-49cd-b10d-a2c75154906f None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:18.211403 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a870c8-e1bd-49cd-b10d-a2c75154906f None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:18.211752 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a870c8-e1bd-49cd-b10d-a2c75154906f None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:18.214721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1349/2682] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 06:38:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:38:18.222974 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da4799-b047-402d-9883-4076ba6f3dca None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:18.223412 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da4799-b047-402d-9883-4076ba6f3dca None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:18.223878 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da4799-b047-402d-9883-4076ba6f3dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:18.270294 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-a4da4799-b047-402d-9883-4076ba6f3dca None None] MFA Rules not processed for user `226f4700a516415cbbd272fdfad7af58`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:18.274558 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a4da4799-b047-402d-9883-4076ba6f3dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:18.290498 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1334/2683] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:18.298121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce1d8f-cc1a-4fc9-9eb3-a970dd3ac521 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:18.299849 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-42ce1d8f-cc1a-4fc9-9eb3-a970dd3ac521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:18.324959 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:18.347002 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=100 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:18.384222 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1350/2684] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 06:38:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:18.385154 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c17c4-0ae3-47e3-921c-a70d24a1479c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:18.386030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c17c4-0ae3-47e3-921c-a70d24a1479c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:18.386313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c17c4-0ae3-47e3-921c-a70d24a1479c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:18.389618 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce1d8f-cc1a-4fc9-9eb3-a970dd3ac521 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:18.395261 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e731cbe-cd2a-499a-b7da-903820fc1f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:18.396148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e731cbe-cd2a-499a-b7da-903820fc1f4e service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:18.396500 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e731cbe-cd2a-499a-b7da-903820fc1f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:18.396962 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e731cbe-cd2a-499a-b7da-903820fc1f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:18.410620 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e731cbe-cd2a-499a-b7da-903820fc1f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:18.412387 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e731cbe-cd2a-499a-b7da-903820fc1f4e service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:18.418012 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-0c6c17c4-0ae3-47e3-921c-a70d24a1479c None None] MFA Rules not processed for user `c1815cc5a2fd43259c5b53650dbc7603`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:18.419447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0c6c17c4-0ae3-47e3-921c-a70d24a1479c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:18.425843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1335/2685] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:38:18.426562 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e731cbe-cd2a-499a-b7da-903820fc1f4e service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:18.436621 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e731cbe-cd2a-499a-b7da-903820fc1f4e service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:18.440802 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9060a8-ca19-490d-afd3-2886acc6c881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:18.441529 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9060a8-ca19-490d-afd3-2886acc6c881 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:18.442006 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9060a8-ca19-490d-afd3-2886acc6c881 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:18.442902 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9060a8-ca19-490d-afd3-2886acc6c881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:18.452469 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9060a8-ca19-490d-afd3-2886acc6c881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:18.453995 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9060a8-ca19-490d-afd3-2886acc6c881 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:18.463937 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1336/2686] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:38:18.514952 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1351/2687] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:18.520504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9060a8-ca19-490d-afd3-2886acc6c881 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:18.523385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ec9060a8-ca19-490d-afd3-2886acc6c881 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:18.550655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9060a8-ca19-490d-afd3-2886acc6c881 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:18.562262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3c757-476c-4a2b-8d2c-b897a7bffd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:18.568959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3c757-476c-4a2b-8d2c-b897a7bffd08 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:18.568959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3c757-476c-4a2b-8d2c-b897a7bffd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:18.568959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3c757-476c-4a2b-8d2c-b897a7bffd08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:18.568959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3c757-476c-4a2b-8d2c-b897a7bffd08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:18.568959 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3c757-476c-4a2b-8d2c-b897a7bffd08 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:18.590159 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49dbf-ebfc-4a2b-b5ba-3009f3d7db94 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:18.591198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49dbf-ebfc-4a2b-b5ba-3009f3d7db94 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:18.591847 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49dbf-ebfc-4a2b-b5ba-3009f3d7db94 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:18.594542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1352/2688] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 06:38:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:38:18.600810 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b27efa-2b67-4a7f-ad3c-b9273bdd7ecc None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:18.601860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b27efa-2b67-4a7f-ad3c-b9273bdd7ecc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:18.602415 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b27efa-2b67-4a7f-ad3c-b9273bdd7ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:18.612056 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1337/2689] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:38:18 2022] POST /identity/v3/projects => generated 379 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:38:18.617994 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3c757-476c-4a2b-8d2c-b897a7bffd08 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:18.646531 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-87b27efa-2b67-4a7f-ad3c-b9273bdd7ecc None None] MFA Rules not processed for user `2a09f4878eac4d60a9a7dcedba15c05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:18.648276 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-87b27efa-2b67-4a7f-ad3c-b9273bdd7ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:18.788121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3c757-476c-4a2b-8d2c-b897a7bffd08 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:18.793449 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9371c27-e292-4b79-a61e-7089f07ecb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:18.794148 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9371c27-e292-4b79-a61e-7089f07ecb5e admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:18.795347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9371c27-e292-4b79-a61e-7089f07ecb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:18.795347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9371c27-e292-4b79-a61e-7089f07ecb5e admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:18.796199 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9371c27-e292-4b79-a61e-7089f07ecb5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:18.798213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9371c27-e292-4b79-a61e-7089f07ecb5e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:18.799430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1353/2690] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 06:38:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:18.812634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b27efa-2b67-4a7f-ad3c-b9273bdd7ecc None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:18.830810 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1338/2691] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:38:18 2022] POST /identity/v3/users => generated 296 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:38:18.836065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9371c27-e292-4b79-a61e-7089f07ecb5e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:18.854479 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9371c27-e292-4b79-a61e-7089f07ecb5e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:18.860767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb1143-3999-413a-8fb3-18690a4759ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:18.861514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb1143-3999-413a-8fb3-18690a4759ea admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:18.861661 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb1143-3999-413a-8fb3-18690a4759ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:18.861975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb1143-3999-413a-8fb3-18690a4759ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:18.862793 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb1143-3999-413a-8fb3-18690a4759ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:18.864610 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb1143-3999-413a-8fb3-18690a4759ea admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:18.890562 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1339/2692] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:38:18.895446 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb1143-3999-413a-8fb3-18690a4759ea admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:18.921920 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb1143-3999-413a-8fb3-18690a4759ea admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:18.928043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f2df8-7e7a-4e57-b801-aff1ecda9857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:18.929049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f2df8-7e7a-4e57-b801-aff1ecda9857 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:18.929377 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f2df8-7e7a-4e57-b801-aff1ecda9857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:18.929665 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f2df8-7e7a-4e57-b801-aff1ecda9857 admin admin] PATH_INFO: `/v3/projects/8aaacebf8e2d4c219ad3adf708ac7dd5/users/bd6c775c3efc49cd8f0000f57f96a24b/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:18.949703 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f2df8-7e7a-4e57-b801-aff1ecda9857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aaacebf8e2d4c219ad3adf708ac7dd5, user_id=bd6c775c3efc49cd8f0000f57f96a24b, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:18.953190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f2df8-7e7a-4e57-b801-aff1ecda9857 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:18.954334 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b27efa-2b67-4a7f-ad3c-b9273bdd7ecc None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:18.961870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453c478-b342-4cc0-a8a8-ec405d7e6638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:18.963041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453c478-b342-4cc0-a8a8-ec405d7e6638 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:18.964203 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453c478-b342-4cc0-a8a8-ec405d7e6638 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:18.964712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453c478-b342-4cc0-a8a8-ec405d7e6638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:18.973804 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1340/2693] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:38:18 2022] PUT /identity/v3/projects/8aaacebf8e2d4c219ad3adf708ac7dd5/users/bd6c775c3efc49cd8f0000f57f96a24b/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:38:18.974823 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f2df8-7e7a-4e57-b801-aff1ecda9857 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:18.978425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a55f2df8-7e7a-4e57-b801-aff1ecda9857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:18.987780 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453c478-b342-4cc0-a8a8-ec405d7e6638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:18.989906 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453c478-b342-4cc0-a8a8-ec405d7e6638 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:18.998280 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:19.005341 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1354/2694] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:38:19.006804 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453c478-b342-4cc0-a8a8-ec405d7e6638 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:19.020263 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453c478-b342-4cc0-a8a8-ec405d7e6638 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:19.066921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:19.096854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29152879-0411-47be-a5b4-6e6d6903f5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:19.097615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29152879-0411-47be-a5b4-6e6d6903f5ee admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:19.097953 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29152879-0411-47be-a5b4-6e6d6903f5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:19.098356 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29152879-0411-47be-a5b4-6e6d6903f5ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:19.098943 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29152879-0411-47be-a5b4-6e6d6903f5ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:19.100777 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29152879-0411-47be-a5b4-6e6d6903f5ee admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:19.102250 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f2df8-7e7a-4e57-b801-aff1ecda9857 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:19.118073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b63e7-6c09-4c07-873b-beec7d40f60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:19.118996 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b63e7-6c09-4c07-873b-beec7d40f60a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:19.119294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b63e7-6c09-4c07-873b-beec7d40f60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:19.119638 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b63e7-6c09-4c07-873b-beec7d40f60a admin admin] PATH_INFO: `/v3/projects/631e7f55ea434c3f9ddbb9ec32e232c3` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:19.120453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1355/2695] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:38:19.125531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b63e7-6c09-4c07-873b-beec7d40f60a admin admin] RBAC: Authorizing `identity:delete_project(project_id=631e7f55ea434c3f9ddbb9ec32e232c3)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:19.126965 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29152879-0411-47be-a5b4-6e6d6903f5ee admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:19.127349 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b63e7-6c09-4c07-873b-beec7d40f60a admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:19.140299 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29152879-0411-47be-a5b4-6e6d6903f5ee admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:19.146541 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e647b0e-b9dd-47e6-b362-5d0abc05ca87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:19.147556 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e647b0e-b9dd-47e6-b362-5d0abc05ca87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:19.147762 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e647b0e-b9dd-47e6-b362-5d0abc05ca87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:19.149191 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e647b0e-b9dd-47e6-b362-5d0abc05ca87 admin admin] PATH_INFO: `/v3/projects/8aaacebf8e2d4c219ad3adf708ac7dd5/users/bd6c775c3efc49cd8f0000f57f96a24b/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:19.150693 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e647b0e-b9dd-47e6-b362-5d0abc05ca87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aaacebf8e2d4c219ad3adf708ac7dd5, user_id=bd6c775c3efc49cd8f0000f57f96a24b, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:19.154599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e647b0e-b9dd-47e6-b362-5d0abc05ca87 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:19.177339 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1356/2696] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:38:19 2022] PUT /identity/v3/projects/8aaacebf8e2d4c219ad3adf708ac7dd5/users/bd6c775c3efc49cd8f0000f57f96a24b/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:19.253458 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7e2b63e7-6c09-4c07-873b-beec7d40f60a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '631e7f55ea434c3f9ddbb9ec32e232c3', 'request_id': 'req-7e2b63e7-6c09-4c07-873b-beec7d40f60a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:19.270064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7e2b63e7-6c09-4c07-873b-beec7d40f60a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '631e7f55ea434c3f9ddbb9ec32e232c3', 'request_id': 'req-7e2b63e7-6c09-4c07-873b-beec7d40f60a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:19.280737 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7e2b63e7-6c09-4c07-873b-beec7d40f60a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '631e7f55ea434c3f9ddbb9ec32e232c3', 'request_id': 'req-7e2b63e7-6c09-4c07-873b-beec7d40f60a'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:19.302437 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7e2b63e7-6c09-4c07-873b-beec7d40f60a admin admin] The token cache is being invalidate because project 631e7f55ea434c3f9ddbb9ec32e232c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:38:19.302929 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7e2b63e7-6c09-4c07-873b-beec7d40f60a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:19.303554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7e2b63e7-6c09-4c07-873b-beec7d40f60a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:38:19.311099 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1341/2697] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:38:18 2022] DELETE /identity/v3/projects/631e7f55ea434c3f9ddbb9ec32e232c3 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:38:19.314322 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd3f37-85fd-48b1-9f49-71f1af35196e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:19.314753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd3f37-85fd-48b1-9f49-71f1af35196e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:19.315978 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd3f37-85fd-48b1-9f49-71f1af35196e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:19.347346 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-00bd3f37-85fd-48b1-9f49-71f1af35196e None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:19.348944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-00bd3f37-85fd-48b1-9f49-71f1af35196e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:19.491196 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1357/2698] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:19.494088 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fbd8c-242d-463a-b9e7-31fc546ee6c4 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:19.494720 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fbd8c-242d-463a-b9e7-31fc546ee6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:19.495244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fbd8c-242d-463a-b9e7-31fc546ee6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:19.520846 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-6a5fbd8c-242d-463a-b9e7-31fc546ee6c4 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:19.523216 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6a5fbd8c-242d-463a-b9e7-31fc546ee6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:19.548925 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1342/2699] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:38:19.553741 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd3f37-85fd-48b1-9f49-71f1af35196e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:19.568185 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd3f37-85fd-48b1-9f49-71f1af35196e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:19.579234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d48b56-da6f-4b5d-baf8-4b1871d38af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:19.580077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d48b56-da6f-4b5d-baf8-4b1871d38af9 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:19.580365 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d48b56-da6f-4b5d-baf8-4b1871d38af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:19.580775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d48b56-da6f-4b5d-baf8-4b1871d38af9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:19.582054 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d48b56-da6f-4b5d-baf8-4b1871d38af9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:38:19.582370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d48b56-da6f-4b5d-baf8-4b1871d38af9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:19.584131 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d48b56-da6f-4b5d-baf8-4b1871d38af9 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:19.600364 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1358/2700] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 06:38:19.609072 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fbd8c-242d-463a-b9e7-31fc546ee6c4 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:19.638285 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fbd8c-242d-463a-b9e7-31fc546ee6c4 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:19.651127 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d8f51-ae6f-4ccb-93f3-426f0f980711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:19.653313 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d8f51-ae6f-4ccb-93f3-426f0f980711 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:19.653657 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d8f51-ae6f-4ccb-93f3-426f0f980711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:19.653952 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d8f51-ae6f-4ccb-93f3-426f0f980711 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:19.655043 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d8f51-ae6f-4ccb-93f3-426f0f980711 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:19.660589 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d8f51-ae6f-4ccb-93f3-426f0f980711 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:19.710934 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d48b56-da6f-4b5d-baf8-4b1871d38af9 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:19.719159 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1343/2701] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:38:19 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:38:19.743257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d48b56-da6f-4b5d-baf8-4b1871d38af9 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:19.840936 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258c3e9-7efe-499a-9874-289c54ec2ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:19.841817 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258c3e9-7efe-499a-9874-289c54ec2ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:19.842114 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258c3e9-7efe-499a-9874-289c54ec2ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:19.842398 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258c3e9-7efe-499a-9874-289c54ec2ea0 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:19.843117 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258c3e9-7efe-499a-9874-289c54ec2ea0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:19.846553 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258c3e9-7efe-499a-9874-289c54ec2ea0 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:19.872601 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:19.886028 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1359/2702] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:38:19 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:38:19.890170 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d8f51-ae6f-4ccb-93f3-426f0f980711 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:19.904284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d8f51-ae6f-4ccb-93f3-426f0f980711 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:19.915639 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373701b6-f21e-47fe-a30c-30a898e4f43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:19.916498 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373701b6-f21e-47fe-a30c-30a898e4f43d admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:19.919809 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373701b6-f21e-47fe-a30c-30a898e4f43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:19.920244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373701b6-f21e-47fe-a30c-30a898e4f43d admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:19.921122 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373701b6-f21e-47fe-a30c-30a898e4f43d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:19.924375 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373701b6-f21e-47fe-a30c-30a898e4f43d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:19.960578 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1344/2703] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:38:19.960790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258c3e9-7efe-499a-9874-289c54ec2ea0 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:19.974183 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258c3e9-7efe-499a-9874-289c54ec2ea0 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:19.981840 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9a261-407b-4f34-bd6e-01cd246ece92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:19.982867 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9a261-407b-4f34-bd6e-01cd246ece92 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:19.983228 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9a261-407b-4f34-bd6e-01cd246ece92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:19.983592 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9a261-407b-4f34-bd6e-01cd246ece92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:19.984870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9a261-407b-4f34-bd6e-01cd246ece92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:19.987005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9a261-407b-4f34-bd6e-01cd246ece92 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:20.002586 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1360/2704] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:38:20.008957 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373701b6-f21e-47fe-a30c-30a898e4f43d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:20.031224 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373701b6-f21e-47fe-a30c-30a898e4f43d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:20.045187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd1d5e-57b7-40d4-94b8-fa6e0458a8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:20.046941 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd1d5e-57b7-40d4-94b8-fa6e0458a8da admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:20.047448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd1d5e-57b7-40d4-94b8-fa6e0458a8da admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:20.048336 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd1d5e-57b7-40d4-94b8-fa6e0458a8da admin admin] PATH_INFO: `/v3/projects/887496db011b48b5a8a5f2da6d8c5e44/users/87cde6f1714941f9a0a094332eae9404/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:20.089823 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd1d5e-57b7-40d4-94b8-fa6e0458a8da admin admin] RBAC: Authorizing `identity:create_grant(project_id=887496db011b48b5a8a5f2da6d8c5e44, user_id=87cde6f1714941f9a0a094332eae9404, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:20.096119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd1d5e-57b7-40d4-94b8-fa6e0458a8da admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:20.147126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed12e7-7829-4d45-9536-2a7cd2825401 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:20.147895 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed12e7-7829-4d45-9536-2a7cd2825401 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:20.149054 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed12e7-7829-4d45-9536-2a7cd2825401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:20.153324 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1345/2705] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:38:20 2022] PUT /identity/v3/projects/887496db011b48b5a8a5f2da6d8c5e44/users/87cde6f1714941f9a0a094332eae9404/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:38:20.186022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ffed12e7-7829-4d45-9536-2a7cd2825401 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:20.187812 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ffed12e7-7829-4d45-9536-2a7cd2825401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:20.309014 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1361/2706] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:20.378123 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd1d5e-57b7-40d4-94b8-fa6e0458a8da admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:20.381537 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e7bd1d5e-57b7-40d4-94b8-fa6e0458a8da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:20.572030 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd1d5e-57b7-40d4-94b8-fa6e0458a8da admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:20.582327 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e7f6a-7ff5-4113-9696-b096a44bd124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:20.583918 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e7f6a-7ff5-4113-9696-b096a44bd124 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:20.584514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e7f6a-7ff5-4113-9696-b096a44bd124 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:20.585456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e7f6a-7ff5-4113-9696-b096a44bd124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:20.600012 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e7f6a-7ff5-4113-9696-b096a44bd124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:20.602454 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e7f6a-7ff5-4113-9696-b096a44bd124 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:20.617555 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1346/2707] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:38:22.731868 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5aaf9c-f133-401a-b27c-52e2b5e4af52 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:22.732390 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5aaf9c-f133-401a-b27c-52e2b5e4af52 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:22.732511 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5aaf9c-f133-401a-b27c-52e2b5e4af52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:22.833138 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-5c5aaf9c-f133-401a-b27c-52e2b5e4af52 None None] MFA Rules not processed for user `87cde6f1714941f9a0a094332eae9404`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:22.841980 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-5c5aaf9c-f133-401a-b27c-52e2b5e4af52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:22.978432 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1362/2708] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:22.988298 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e7f6a-7ff5-4113-9696-b096a44bd124 service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:22.990628 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-a65e7f6a-7ff5-4113-9696-b096a44bd124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:23.022403 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:23.109992 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e7f6a-7ff5-4113-9696-b096a44bd124 service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:23.115753 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63cfeb-ef7b-4687-a0ff-974e9f7fef8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:23.116830 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63cfeb-ef7b-4687-a0ff-974e9f7fef8f service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:23.117478 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63cfeb-ef7b-4687-a0ff-974e9f7fef8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:23.118412 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63cfeb-ef7b-4687-a0ff-974e9f7fef8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:23.141941 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63cfeb-ef7b-4687-a0ff-974e9f7fef8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:23.144576 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63cfeb-ef7b-4687-a0ff-974e9f7fef8f service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:23.158401 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1347/2709] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:38:23.189352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc5363-c80e-4bce-aa44-e7ff6fc26fcd None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:23.189705 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc5363-c80e-4bce-aa44-e7ff6fc26fcd None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:23.190021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc5363-c80e-4bce-aa44-e7ff6fc26fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:23.236474 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ebfc5363-c80e-4bce-aa44-e7ff6fc26fcd None None] MFA Rules not processed for user `aee77a3ed91445ab8b7f58e7f8bc348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:23.243534 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ebfc5363-c80e-4bce-aa44-e7ff6fc26fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:23.422880 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63cfeb-ef7b-4687-a0ff-974e9f7fef8f service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:23.423896 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1363/2710] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 06:38:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:23.425042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-aa63cfeb-ef7b-4687-a0ff-974e9f7fef8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:23.552825 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63cfeb-ef7b-4687-a0ff-974e9f7fef8f service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:23.565540 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd8085-e0bb-4c55-bd5b-c6c0864722d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc205d053d54506bb6c9227e4a1b044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:23.566833 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd8085-e0bb-4c55-bd5b-c6c0864722d0 service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:23.568026 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd8085-e0bb-4c55-bd5b-c6c0864722d0 service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:23.568531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd8085-e0bb-4c55-bd5b-c6c0864722d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:23.591369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd8085-e0bb-4c55-bd5b-c6c0864722d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:23.593933 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd8085-e0bb-4c55-bd5b-c6c0864722d0 service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:23.677898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1348/2711] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 06:38:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 06:38:24.970689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793fbd0-d1bb-41b7-84ec-741ee40ae530 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:24.971495 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793fbd0-d1bb-41b7-84ec-741ee40ae530 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:24.973032 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793fbd0-d1bb-41b7-84ec-741ee40ae530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:25.039173 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f793fbd0-d1bb-41b7-84ec-741ee40ae530 None None] MFA Rules not processed for user `87cde6f1714941f9a0a094332eae9404`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:25.047513 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f793fbd0-d1bb-41b7-84ec-741ee40ae530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:25.086908 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1364/2712] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:27.414275 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0be33-190d-44fa-bfff-5549f9bc58f4 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:27.414850 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0be33-190d-44fa-bfff-5549f9bc58f4 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:27.414850 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0be33-190d-44fa-bfff-5549f9bc58f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:27.467000 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-bea0be33-190d-44fa-bfff-5549f9bc58f4 None None] MFA Rules not processed for user `bd6c775c3efc49cd8f0000f57f96a24b`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:27.468275 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bea0be33-190d-44fa-bfff-5549f9bc58f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:27.602733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1349/2713] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:27.610093 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793fbd0-d1bb-41b7-84ec-741ee40ae530 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:27.613037 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f793fbd0-d1bb-41b7-84ec-741ee40ae530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:27.661231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:27.783981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793fbd0-d1bb-41b7-84ec-741ee40ae530 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:27.806828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacc0f2-e1b0-4f14-89ba-e3d0f93747cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:27.807933 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacc0f2-e1b0-4f14-89ba-e3d0f93747cb service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:27.808291 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacc0f2-e1b0-4f14-89ba-e3d0f93747cb service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:27.808641 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacc0f2-e1b0-4f14-89ba-e3d0f93747cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:27.813723 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-deacc0f2-e1b0-4f14-89ba-e3d0f93747cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:27.948984 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deacc0f2-e1b0-4f14-89ba-e3d0f93747cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:27.956174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deacc0f2-e1b0-4f14-89ba-e3d0f93747cb service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:27.989087 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1365/2714] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:38:37.583709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=84 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:37.962093 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=48 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:40.584549 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0be33-190d-44fa-bfff-5549f9bc58f4 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:40.586068 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bea0be33-190d-44fa-bfff-5549f9bc58f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:40.656061 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0be33-190d-44fa-bfff-5549f9bc58f4 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:40.660973 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08cc40-99e8-475d-9f0f-54c4bc27eb6c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b76876d821e4bbbb8d5cad5bd26f559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:40.661657 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08cc40-99e8-475d-9f0f-54c4bc27eb6c service placement] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:40.661944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08cc40-99e8-475d-9f0f-54c4bc27eb6c service placement] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:40.662223 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08cc40-99e8-475d-9f0f-54c4bc27eb6c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:40.663994 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4c08cc40-99e8-475d-9f0f-54c4bc27eb6c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:40.675468 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08cc40-99e8-475d-9f0f-54c4bc27eb6c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:40.677036 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08cc40-99e8-475d-9f0f-54c4bc27eb6c service placement] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:40.687088 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1350/2715] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:38:44.370602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c08ad-50d6-456f-b63c-60e68dfc357e None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:44.371233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c08ad-50d6-456f-b63c-60e68dfc357e None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:44.371233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c08ad-50d6-456f-b63c-60e68dfc357e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:44.388344 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-149c08ad-50d6-456f-b63c-60e68dfc357e None None] MFA Rules not processed for user `c1815cc5a2fd43259c5b53650dbc7603`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:44.389609 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-149c08ad-50d6-456f-b63c-60e68dfc357e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:44.396904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1366/2716] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:38:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:44.915335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d2676-059f-4d2c-aca6-ba7d16ae7faf None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:44.915846 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d2676-059f-4d2c-aca6-ba7d16ae7faf None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:44.915973 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d2676-059f-4d2c-aca6-ba7d16ae7faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:44.936747 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4a3d2676-059f-4d2c-aca6-ba7d16ae7faf None None] MFA Rules not processed for user `aee77a3ed91445ab8b7f58e7f8bc348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:38:44.938013 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4a3d2676-059f-4d2c-aca6-ba7d16ae7faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:44.952193 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1351/2717] 10.222.0.65 () {58 vars in 967 bytes} [Thu May 19 06:38:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:38:44.967034 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c08ad-50d6-456f-b63c-60e68dfc357e None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:44.979900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c08ad-50d6-456f-b63c-60e68dfc357e None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:44.986533 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689772d-61c7-438b-afa5-056a4736bb03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc205d053d54506bb6c9227e4a1b044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:44.987542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689772d-61c7-438b-afa5-056a4736bb03 service swift] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:44.988063 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689772d-61c7-438b-afa5-056a4736bb03 service swift] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:44.988442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689772d-61c7-438b-afa5-056a4736bb03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:44.998630 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689772d-61c7-438b-afa5-056a4736bb03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:45.000449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689772d-61c7-438b-afa5-056a4736bb03 service swift] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:45.009454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1367/2718] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 06:38:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 06:38:54.929757 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=35 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:55.005953 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=9 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:38:59.803578 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d2676-059f-4d2c-aca6-ba7d16ae7faf None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:38:59.805153 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4a3d2676-059f-4d2c-aca6-ba7d16ae7faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:38:59.819271 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d2676-059f-4d2c-aca6-ba7d16ae7faf None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:38:59.824643 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6cf85-7586-4583-b5c3-7fd038a82538 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:38:59.825713 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6cf85-7586-4583-b5c3-7fd038a82538 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:38:59.826204 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6cf85-7586-4583-b5c3-7fd038a82538 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:38:59.827186 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6cf85-7586-4583-b5c3-7fd038a82538 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:38:59.840460 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6cf85-7586-4583-b5c3-7fd038a82538 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:38:59.842457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6cf85-7586-4583-b5c3-7fd038a82538 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:38:59.856645 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1352/2719] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:38:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:39:09.849466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=6 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:39:33.593076 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897f2db-c4ab-4249-948d-25d116f33bb8 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:39:33.594219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897f2db-c4ab-4249-948d-25d116f33bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:39:33.594344 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897f2db-c4ab-4249-948d-25d116f33bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:39:33.613020 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d897f2db-c4ab-4249-948d-25d116f33bb8 None None] MFA Rules not processed for user `87cde6f1714941f9a0a094332eae9404`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:39:33.614281 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d897f2db-c4ab-4249-948d-25d116f33bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:39:33.622649 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1368/2720] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:39:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:39:38.156508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aea89-d908-4dc1-acb1-8fea5bf0c9e2 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:39:38.157059 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aea89-d908-4dc1-acb1-8fea5bf0c9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:39:38.157183 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aea89-d908-4dc1-acb1-8fea5bf0c9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:39:38.175774 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-ac9aea89-d908-4dc1-acb1-8fea5bf0c9e2 None None] MFA Rules not processed for user `3a8abbf889e149f5be25ecddb22384a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:39:38.176971 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ac9aea89-d908-4dc1-acb1-8fea5bf0c9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:39:38.231579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1353/2721] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:39:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:39:43.607215 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:39:48.224944 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=25 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:40:28.437672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897f2db-c4ab-4249-948d-25d116f33bb8 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:40:28.448975 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897f2db-c4ab-4249-948d-25d116f33bb8 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:40:28.454109 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef1766-5a6e-46f6-b29d-5fe72ba9db2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:40:28.455059 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef1766-5a6e-46f6-b29d-5fe72ba9db2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:40:28.455342 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef1766-5a6e-46f6-b29d-5fe72ba9db2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:40:28.455616 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef1766-5a6e-46f6-b29d-5fe72ba9db2d admin admin] PATH_INFO: `/v3/users/87cde6f1714941f9a0a094332eae9404` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:40:28.456801 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef1766-5a6e-46f6-b29d-5fe72ba9db2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=87cde6f1714941f9a0a094332eae9404)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:40:28.458760 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef1766-5a6e-46f6-b29d-5fe72ba9db2d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:40:28.508007 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ffef1766-5a6e-46f6-b29d-5fe72ba9db2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87cde6f1714941f9a0a094332eae9404'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:40:28.509144 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ffef1766-5a6e-46f6-b29d-5fe72ba9db2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87cde6f1714941f9a0a094332eae9404'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:40:28.528964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ffef1766-5a6e-46f6-b29d-5fe72ba9db2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87cde6f1714941f9a0a094332eae9404'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:40:28.534964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ffef1766-5a6e-46f6-b29d-5fe72ba9db2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87cde6f1714941f9a0a094332eae9404'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:40:28.543657 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ffef1766-5a6e-46f6-b29d-5fe72ba9db2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87cde6f1714941f9a0a094332eae9404'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:40:28.562287 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1369/2722] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:40:28 2022] DELETE /identity/v3/users/87cde6f1714941f9a0a094332eae9404 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:40:28.733470 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9aea89-d908-4dc1-acb1-8fea5bf0c9e2 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:40:28.744912 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9aea89-d908-4dc1-acb1-8fea5bf0c9e2 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:40:28.810410 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:40:28.811274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:40:28.811555 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:40:28.811829 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df admin admin] PATH_INFO: `/v3/projects/887496db011b48b5a8a5f2da6d8c5e44` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:40:28.813603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df admin admin] RBAC: Authorizing `identity:delete_project(project_id=887496db011b48b5a8a5f2da6d8c5e44)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:40:28.815698 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:40:28.862262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '887496db011b48b5a8a5f2da6d8c5e44', 'request_id': 'req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:40:28.869554 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '887496db011b48b5a8a5f2da6d8c5e44', 'request_id': 'req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:40:28.875746 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '887496db011b48b5a8a5f2da6d8c5e44', 'request_id': 'req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:40:28.887167 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df admin admin] The token cache is being invalidate because project 887496db011b48b5a8a5f2da6d8c5e44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:40:28.887700 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:40:28.888338 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:40:28.889909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1354/2723] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:40:28 2022] DELETE /identity/v3/projects/887496db011b48b5a8a5f2da6d8c5e44 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:40:37.884143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef1766-5a6e-46f6-b29d-5fe72ba9db2d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:40:37.885691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ffef1766-5a6e-46f6-b29d-5fe72ba9db2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:40:37.945881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef1766-5a6e-46f6-b29d-5fe72ba9db2d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:40:37.995747 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31141652-a564-409f-8515-f014d61777d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:40:37.996406 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31141652-a564-409f-8515-f014d61777d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:40:37.996696 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31141652-a564-409f-8515-f014d61777d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:40:37.996978 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31141652-a564-409f-8515-f014d61777d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:40:37.998820 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-31141652-a564-409f-8515-f014d61777d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:40:38.041270 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:40:38.070539 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31141652-a564-409f-8515-f014d61777d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:40:38.071982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31141652-a564-409f-8515-f014d61777d1 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:40:38.083555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1370/2724] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:40:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:40:38.883984 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:40:39.266820 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:40:39.268198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:40:39.280157 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e4803-6a8e-4ac0-b9f2-7f89963802df admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:40:39.283962 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c774d-074c-42de-8c92-f274ca57b27b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:40:39.284584 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c774d-074c-42de-8c92-f274ca57b27b service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:40:39.285422 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c774d-074c-42de-8c92-f274ca57b27b service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:40:39.285686 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c774d-074c-42de-8c92-f274ca57b27b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:40:39.287399 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1f4c774d-074c-42de-8c92-f274ca57b27b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:40:39.301255 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c774d-074c-42de-8c92-f274ca57b27b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:40:39.302706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c774d-074c-42de-8c92-f274ca57b27b service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:40:39.313632 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1355/2725] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:40:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:40:42.114358 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfff871-53e8-408d-8380-66e8e3b4b380 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:40:42.114904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfff871-53e8-408d-8380-66e8e3b4b380 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:40:42.114904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfff871-53e8-408d-8380-66e8e3b4b380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:40:42.132979 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-adfff871-53e8-408d-8380-66e8e3b4b380 None None] MFA Rules not processed for user `aee77a3ed91445ab8b7f58e7f8bc348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:40:42.134217 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-adfff871-53e8-408d-8380-66e8e3b4b380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:40:42.190881 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1371/2726] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 06:40:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:40:42.237086 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c774d-074c-42de-8c92-f274ca57b27b service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:40:42.239234 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1f4c774d-074c-42de-8c92-f274ca57b27b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:40:42.310752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c774d-074c-42de-8c92-f274ca57b27b service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:40:42.316345 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffeb98-e71e-4827-8e4f-8088e3934996 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc205d053d54506bb6c9227e4a1b044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:40:42.316997 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffeb98-e71e-4827-8e4f-8088e3934996 service swift] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:40:42.317302 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffeb98-e71e-4827-8e4f-8088e3934996 service swift] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:40:42.317593 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffeb98-e71e-4827-8e4f-8088e3934996 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:40:42.326256 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffeb98-e71e-4827-8e4f-8088e3934996 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:40:42.327858 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffeb98-e71e-4827-8e4f-8088e3934996 service swift] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:40:42.337010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1356/2727] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 06:40:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 06:40:52.184075 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=39 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:40:52.333233 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=38 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:40:55.903083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64078605-b5b2-47ac-b9b9-acbe85a7c44d None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:40:55.903669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64078605-b5b2-47ac-b9b9-acbe85a7c44d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:40:55.903669 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64078605-b5b2-47ac-b9b9-acbe85a7c44d None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:40:55.905523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1372/2728] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 06:40:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:40:55.911586 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8d628-ee78-425c-b9ee-6f601ec34881 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:40:55.912089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8d628-ee78-425c-b9ee-6f601ec34881 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:40:55.912214 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8d628-ee78-425c-b9ee-6f601ec34881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:40:55.933671 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e9d8d628-ee78-425c-b9ee-6f601ec34881 None None] MFA Rules not processed for user `2a09f4878eac4d60a9a7dcedba15c05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:40:55.934891 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e9d8d628-ee78-425c-b9ee-6f601ec34881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:40:55.945610 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1357/2729] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 06:40:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:40:55.958752 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64078605-b5b2-47ac-b9b9-acbe85a7c44d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:40:55.960599 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-64078605-b5b2-47ac-b9b9-acbe85a7c44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:40:56.022496 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64078605-b5b2-47ac-b9b9-acbe85a7c44d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:40:56.026317 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55f780-6e85-4286-83ba-c88352076dcd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a812c3b9b20f44f8814051381a8005b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:40:56.026969 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55f780-6e85-4286-83ba-c88352076dcd service glance] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:40:56.027250 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55f780-6e85-4286-83ba-c88352076dcd service glance] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:40:56.027524 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55f780-6e85-4286-83ba-c88352076dcd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:40:56.039500 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55f780-6e85-4286-83ba-c88352076dcd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:40:56.040950 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55f780-6e85-4286-83ba-c88352076dcd service glance] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:40:56.052432 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1373/2730] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:40:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:40:56.119873 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8b82a-b5a9-45db-a144-390ebbe2092f None None] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:40:56.120169 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8b82a-b5a9-45db-a144-390ebbe2092f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:40:56.120456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8b82a-b5a9-45db-a144-390ebbe2092f None None] PATH_INFO: `/` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:40:56.121956 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1358/2731] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 06:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:40:56.126521 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0943928-dd69-4a2c-ae71-7e966f4ab232 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:40:56.126813 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0943928-dd69-4a2c-ae71-7e966f4ab232 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:40:56.127095 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0943928-dd69-4a2c-ae71-7e966f4ab232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:40:56.163774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-d0943928-dd69-4a2c-ae71-7e966f4ab232 None None] MFA Rules not processed for user `226f4700a516415cbbd272fdfad7af58`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:40:56.164989 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-d0943928-dd69-4a2c-ae71-7e966f4ab232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:40:56.171560 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1374/2732] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 06:40:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:40:56.184260 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8b82a-b5a9-45db-a144-390ebbe2092f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:40:56.186025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-46b8b82a-b5a9-45db-a144-390ebbe2092f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:40:56.199858 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8b82a-b5a9-45db-a144-390ebbe2092f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:40:56.203457 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da042844-e5c5-430a-8bf4-787a03f4e001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:40:56.204096 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da042844-e5c5-430a-8bf4-787a03f4e001 service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:40:56.204389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da042844-e5c5-430a-8bf4-787a03f4e001 service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:40:56.204670 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da042844-e5c5-430a-8bf4-787a03f4e001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:40:56.212025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da042844-e5c5-430a-8bf4-787a03f4e001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:40:56.213501 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da042844-e5c5-430a-8bf4-787a03f4e001 service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:40:56.221177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1359/2733] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:40:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:40:56.303650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0579541-baba-4e19-98a4-c190ca633378 None None] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:40:56.303940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0579541-baba-4e19-98a4-c190ca633378 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:40:56.304222 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0579541-baba-4e19-98a4-c190ca633378 None None] PATH_INFO: `/` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:40:56.305696 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1375/2734] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 06:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 06:40:56.310717 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d5cd5-1432-4cd2-93a7-60e1a625230e None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:40:56.310999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d5cd5-1432-4cd2-93a7-60e1a625230e None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:40:56.311276 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d5cd5-1432-4cd2-93a7-60e1a625230e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:40:56.330553 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b16d5cd5-1432-4cd2-93a7-60e1a625230e None None] MFA Rules not processed for user `2a09f4878eac4d60a9a7dcedba15c05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:40:56.331785 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b16d5cd5-1432-4cd2-93a7-60e1a625230e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:40:56.341396 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1360/2735] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 06:40:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:40:56.353206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0579541-baba-4e19-98a4-c190ca633378 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:40:56.365268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0579541-baba-4e19-98a4-c190ca633378 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:40:56.368960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede063b-06fe-4231-a98e-f1ee2ee3db9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:40:56.369650 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede063b-06fe-4231-a98e-f1ee2ee3db9f service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:40:56.369938 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede063b-06fe-4231-a98e-f1ee2ee3db9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:40:56.370220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede063b-06fe-4231-a98e-f1ee2ee3db9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:40:56.381091 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede063b-06fe-4231-a98e-f1ee2ee3db9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:40:56.382602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede063b-06fe-4231-a98e-f1ee2ee3db9f service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:40:56.393998 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1376/2736] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:40:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:40:57.946509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada9b18-6279-4ced-95e0-e5f1d05c2f7f None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:40:57.947040 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada9b18-6279-4ced-95e0-e5f1d05c2f7f None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:40:57.947040 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada9b18-6279-4ced-95e0-e5f1d05c2f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:40:57.967239 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-6ada9b18-6279-4ced-95e0-e5f1d05c2f7f None None] MFA Rules not processed for user `aee77a3ed91445ab8b7f58e7f8bc348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:40:57.968456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6ada9b18-6279-4ced-95e0-e5f1d05c2f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:40:57.975625 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1361/2737] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 06:40:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:40:57.988008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede063b-06fe-4231-a98e-f1ee2ee3db9f service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:40:58.000253 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede063b-06fe-4231-a98e-f1ee2ee3db9f service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:40:58.005812 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1587a-eb00-47ae-aae7-578dc2e03369 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc205d053d54506bb6c9227e4a1b044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:40:58.006460 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1587a-eb00-47ae-aae7-578dc2e03369 service swift] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:40:58.006748 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1587a-eb00-47ae-aae7-578dc2e03369 service swift] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:40:58.007034 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1587a-eb00-47ae-aae7-578dc2e03369 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:40:58.015777 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1587a-eb00-47ae-aae7-578dc2e03369 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:40:58.017417 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1587a-eb00-47ae-aae7-578dc2e03369 service swift] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:40:58.026139 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1377/2738] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 06:40:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 06:41:07.960427 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=15 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:41:08.023054 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=47 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:41:09.342844 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada9b18-6279-4ced-95e0-e5f1d05c2f7f None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:09.345662 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-6ada9b18-6279-4ced-95e0-e5f1d05c2f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:41:09.389567 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada9b18-6279-4ced-95e0-e5f1d05c2f7f None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:09.395464 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3408074-8648-4bd9-a6a8-9750ac816ca9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:09.396508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3408074-8648-4bd9-a6a8-9750ac816ca9 service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:09.397079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3408074-8648-4bd9-a6a8-9750ac816ca9 service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:09.397591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3408074-8648-4bd9-a6a8-9750ac816ca9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:09.400437 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c3408074-8648-4bd9-a6a8-9750ac816ca9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:41:09.417562 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3408074-8648-4bd9-a6a8-9750ac816ca9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:09.420980 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3408074-8648-4bd9-a6a8-9750ac816ca9 service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:09.437198 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1362/2739] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:41:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:41:10.016774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1587a-eb00-47ae-aae7-578dc2e03369 service swift] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:10.027822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1587a-eb00-47ae-aae7-578dc2e03369 service swift] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:10.032707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57910d-dc88-4ac3-bd66-d51668b56d0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:10.033497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57910d-dc88-4ac3-bd66-d51668b56d0b service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:10.033751 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57910d-dc88-4ac3-bd66-d51668b56d0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:10.033968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57910d-dc88-4ac3-bd66-d51668b56d0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:10.036318 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-9c57910d-dc88-4ac3-bd66-d51668b56d0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:41:10.050091 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57910d-dc88-4ac3-bd66-d51668b56d0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:10.051561 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57910d-dc88-4ac3-bd66-d51668b56d0b service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:10.061311 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1378/2740] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:41:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:41:10.521591 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99aa68b-3809-42f2-83c7-e53ee3cc948d None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:10.522232 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99aa68b-3809-42f2-83c7-e53ee3cc948d None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:10.522232 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99aa68b-3809-42f2-83c7-e53ee3cc948d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:10.540129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-e99aa68b-3809-42f2-83c7-e53ee3cc948d None None] MFA Rules not processed for user `c1815cc5a2fd43259c5b53650dbc7603`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:41:10.541352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e99aa68b-3809-42f2-83c7-e53ee3cc948d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:41:10.595433 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1363/2741] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:41:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:41:10.910641 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57910d-dc88-4ac3-bd66-d51668b56d0b service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:10.919928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57910d-dc88-4ac3-bd66-d51668b56d0b service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:10.923542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec1cc8-7893-42c6-a0f5-396b37540fd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:10.924190 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec1cc8-7893-42c6-a0f5-396b37540fd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:10.924473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec1cc8-7893-42c6-a0f5-396b37540fd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:10.924753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec1cc8-7893-42c6-a0f5-396b37540fd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:10.926564 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2eec1cc8-7893-42c6-a0f5-396b37540fd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:41:10.936898 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec1cc8-7893-42c6-a0f5-396b37540fd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:10.938367 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec1cc8-7893-42c6-a0f5-396b37540fd9 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:10.946783 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1379/2742] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:41:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:41:11.151887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682c9cf-0f91-4e1f-a06d-46b12342cb50 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:11.152185 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682c9cf-0f91-4e1f-a06d-46b12342cb50 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:11.152471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682c9cf-0f91-4e1f-a06d-46b12342cb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:11.171058 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-1682c9cf-0f91-4e1f-a06d-46b12342cb50 None None] MFA Rules not processed for user `aee77a3ed91445ab8b7f58e7f8bc348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:41:11.172206 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1682c9cf-0f91-4e1f-a06d-46b12342cb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:41:11.180843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1364/2743] 10.222.0.65 () {58 vars in 967 bytes} [Thu May 19 06:41:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:41:11.193539 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec1cc8-7893-42c6-a0f5-396b37540fd9 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:11.206337 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec1cc8-7893-42c6-a0f5-396b37540fd9 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:11.211792 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb506ee-e6d6-4298-8bda-45f3a3290b3c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fc205d053d54506bb6c9227e4a1b044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:11.212500 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb506ee-e6d6-4298-8bda-45f3a3290b3c service swift] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:11.212871 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb506ee-e6d6-4298-8bda-45f3a3290b3c service swift] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:11.213219 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb506ee-e6d6-4298-8bda-45f3a3290b3c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:11.222233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb506ee-e6d6-4298-8bda-45f3a3290b3c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:11.223809 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb506ee-e6d6-4298-8bda-45f3a3290b3c service swift] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:11.233528 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1380/2744] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 06:41:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 06:41:11.431665 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682c9cf-0f91-4e1f-a06d-46b12342cb50 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:11.433838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-1682c9cf-0f91-4e1f-a06d-46b12342cb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:41:11.529165 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682c9cf-0f91-4e1f-a06d-46b12342cb50 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:11.538933 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c746eb-6bbc-4f02-bf9c-36096fabab6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b76876d821e4bbbb8d5cad5bd26f559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:11.539793 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c746eb-6bbc-4f02-bf9c-36096fabab6b service placement] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:11.540082 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c746eb-6bbc-4f02-bf9c-36096fabab6b service placement] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:11.540359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c746eb-6bbc-4f02-bf9c-36096fabab6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:11.542244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d4c746eb-6bbc-4f02-bf9c-36096fabab6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:41:11.555052 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c746eb-6bbc-4f02-bf9c-36096fabab6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:11.556678 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c746eb-6bbc-4f02-bf9c-36096fabab6b service placement] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:11.570157 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1365/2745] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:41:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:41:21.229089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=18 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:41:21.564131 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=66 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:41:51.156796 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb506ee-e6d6-4298-8bda-45f3a3290b3c service swift] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:51.158517 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-4cb506ee-e6d6-4298-8bda-45f3a3290b3c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:41:51.216490 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb506ee-e6d6-4298-8bda-45f3a3290b3c service swift] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:51.220178 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abdc6b-79a5-435f-97da-7dc6f2b10a56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:51.220852 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abdc6b-79a5-435f-97da-7dc6f2b10a56 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:51.221137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abdc6b-79a5-435f-97da-7dc6f2b10a56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:51.221453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abdc6b-79a5-435f-97da-7dc6f2b10a56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:51.223252 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-01abdc6b-79a5-435f-97da-7dc6f2b10a56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:41:51.283193 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01abdc6b-79a5-435f-97da-7dc6f2b10a56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:51.284633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01abdc6b-79a5-435f-97da-7dc6f2b10a56 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:51.294227 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1381/2746] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:41:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:41:52.588610 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c746eb-6bbc-4f02-bf9c-36096fabab6b service placement] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:52.598182 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c746eb-6bbc-4f02-bf9c-36096fabab6b service placement] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:52.601758 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6dcdf-2dfa-40e0-a2ef-21fed332e4af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:52.603099 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6dcdf-2dfa-40e0-a2ef-21fed332e4af service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:52.603567 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6dcdf-2dfa-40e0-a2ef-21fed332e4af service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:52.603853 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6dcdf-2dfa-40e0-a2ef-21fed332e4af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:52.605733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e1a6dcdf-2dfa-40e0-a2ef-21fed332e4af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:41:52.669857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a6dcdf-2dfa-40e0-a2ef-21fed332e4af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:52.671320 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a6dcdf-2dfa-40e0-a2ef-21fed332e4af service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:52.680977 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1366/2747] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:41:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:41:55.549746 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abdc6b-79a5-435f-97da-7dc6f2b10a56 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:55.551234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-01abdc6b-79a5-435f-97da-7dc6f2b10a56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:41:55.571832 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abdc6b-79a5-435f-97da-7dc6f2b10a56 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:55.577347 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e99f5-4f8f-4cf7-abc6-bae1bd241817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:55.579664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e99f5-4f8f-4cf7-abc6-bae1bd241817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:55.580025 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e99f5-4f8f-4cf7-abc6-bae1bd241817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:55.580359 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e99f5-4f8f-4cf7-abc6-bae1bd241817 admin admin] PATH_INFO: `/v3/users/3a8abbf889e149f5be25ecddb22384a5` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:55.581754 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e99f5-4f8f-4cf7-abc6-bae1bd241817 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a8abbf889e149f5be25ecddb22384a5)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:55.583816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e99f5-4f8f-4cf7-abc6-bae1bd241817 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:55.644323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b42e99f5-4f8f-4cf7-abc6-bae1bd241817 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a8abbf889e149f5be25ecddb22384a5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:41:55.645002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b42e99f5-4f8f-4cf7-abc6-bae1bd241817 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a8abbf889e149f5be25ecddb22384a5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:41:55.657814 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b42e99f5-4f8f-4cf7-abc6-bae1bd241817 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a8abbf889e149f5be25ecddb22384a5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:41:55.658610 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b42e99f5-4f8f-4cf7-abc6-bae1bd241817 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a8abbf889e149f5be25ecddb22384a5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:41:55.666824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b42e99f5-4f8f-4cf7-abc6-bae1bd241817 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a8abbf889e149f5be25ecddb22384a5'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:41:55.681022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1382/2748] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:41:55 2022] DELETE /identity/v3/users/3a8abbf889e149f5be25ecddb22384a5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:41:55.860796 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6dcdf-2dfa-40e0-a2ef-21fed332e4af service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:55.875631 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6dcdf-2dfa-40e0-a2ef-21fed332e4af service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:55.937942 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a12db7-e494-48f8-a726-821d13dd7cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:55.938719 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a12db7-e494-48f8-a726-821d13dd7cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:55.939006 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a12db7-e494-48f8-a726-821d13dd7cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:55.939283 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a12db7-e494-48f8-a726-821d13dd7cbc admin admin] PATH_INFO: `/v3/projects/e21ed8d40dab40828a1fc685ca63ad92` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:55.940452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a12db7-e494-48f8-a726-821d13dd7cbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e21ed8d40dab40828a1fc685ca63ad92)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:55.942426 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a12db7-e494-48f8-a726-821d13dd7cbc admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:55.989260 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-50a12db7-e494-48f8-a726-821d13dd7cbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e21ed8d40dab40828a1fc685ca63ad92', 'request_id': 'req-50a12db7-e494-48f8-a726-821d13dd7cbc'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:41:55.997084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-50a12db7-e494-48f8-a726-821d13dd7cbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e21ed8d40dab40828a1fc685ca63ad92', 'request_id': 'req-50a12db7-e494-48f8-a726-821d13dd7cbc'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:41:56.002694 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-50a12db7-e494-48f8-a726-821d13dd7cbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e21ed8d40dab40828a1fc685ca63ad92', 'request_id': 'req-50a12db7-e494-48f8-a726-821d13dd7cbc'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:41:56.014011 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-50a12db7-e494-48f8-a726-821d13dd7cbc admin admin] The token cache is being invalidate because project e21ed8d40dab40828a1fc685ca63ad92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:41:56.014296 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-50a12db7-e494-48f8-a726-821d13dd7cbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:41:56.014930 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-50a12db7-e494-48f8-a726-821d13dd7cbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:41:56.016430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1367/2749] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:41:55 2022] DELETE /identity/v3/projects/e21ed8d40dab40828a1fc685ca63ad92 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:41:56.021964 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28bae9-3ac0-4cd0-8c74-0dcd5ff0a25c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:56.022263 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28bae9-3ac0-4cd0-8c74-0dcd5ff0a25c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:56.022552 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28bae9-3ac0-4cd0-8c74-0dcd5ff0a25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:56.042165 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-2a28bae9-3ac0-4cd0-8c74-0dcd5ff0a25c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:41:56.043385 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2a28bae9-3ac0-4cd0-8c74-0dcd5ff0a25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:41:56.065002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:41:56.107085 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1383/2750] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:41:56.114885 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddff2e4-5ddb-40d7-86ba-dcc3cefb8516 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:56.115180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddff2e4-5ddb-40d7-86ba-dcc3cefb8516 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:56.115464 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddff2e4-5ddb-40d7-86ba-dcc3cefb8516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:56.134690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-3ddff2e4-5ddb-40d7-86ba-dcc3cefb8516 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:41:56.135895 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3ddff2e4-5ddb-40d7-86ba-dcc3cefb8516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:41:56.143988 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1368/2751] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:41:56.147944 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28bae9-3ac0-4cd0-8c74-0dcd5ff0a25c None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:56.157852 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28bae9-3ac0-4cd0-8c74-0dcd5ff0a25c None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:56.162356 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be4459-bc98-43f6-90da-897fdac3de0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:56.163009 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be4459-bc98-43f6-90da-897fdac3de0b admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:56.163288 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be4459-bc98-43f6-90da-897fdac3de0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:56.163568 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be4459-bc98-43f6-90da-897fdac3de0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:56.164288 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be4459-bc98-43f6-90da-897fdac3de0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:41:56.164600 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be4459-bc98-43f6-90da-897fdac3de0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:56.166231 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be4459-bc98-43f6-90da-897fdac3de0b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:56.177628 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1384/2752] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:41:56.182536 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddff2e4-5ddb-40d7-86ba-dcc3cefb8516 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:56.191729 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddff2e4-5ddb-40d7-86ba-dcc3cefb8516 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:56.197100 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4751cd6-01ea-4e27-9351-4db06aa47afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:56.197834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4751cd6-01ea-4e27-9351-4db06aa47afd admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:56.198121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4751cd6-01ea-4e27-9351-4db06aa47afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:56.198398 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4751cd6-01ea-4e27-9351-4db06aa47afd admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:56.199083 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4751cd6-01ea-4e27-9351-4db06aa47afd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:56.200978 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4751cd6-01ea-4e27-9351-4db06aa47afd admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:56.215025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:41:56.228793 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1369/2753] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:41:56 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:41:56.232970 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be4459-bc98-43f6-90da-897fdac3de0b admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:56.243343 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be4459-bc98-43f6-90da-897fdac3de0b admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:56.298435 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4f799-5600-4a3d-a52e-27d7bcce7e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:56.299100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4f799-5600-4a3d-a52e-27d7bcce7e29 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:56.299381 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4f799-5600-4a3d-a52e-27d7bcce7e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:56.299652 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4f799-5600-4a3d-a52e-27d7bcce7e29 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:56.300345 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4f799-5600-4a3d-a52e-27d7bcce7e29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:56.302293 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4f799-5600-4a3d-a52e-27d7bcce7e29 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:56.324255 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1385/2754] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:41:56 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:41:56.328381 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4751cd6-01ea-4e27-9351-4db06aa47afd admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:56.339562 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4751cd6-01ea-4e27-9351-4db06aa47afd admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:56.344218 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8939b2-2d90-4582-af80-4ee06100cf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:56.344887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8939b2-2d90-4582-af80-4ee06100cf7e admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:56.345175 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8939b2-2d90-4582-af80-4ee06100cf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:56.345505 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8939b2-2d90-4582-af80-4ee06100cf7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:56.346228 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8939b2-2d90-4582-af80-4ee06100cf7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:56.347839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8939b2-2d90-4582-af80-4ee06100cf7e admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:56.358372 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1370/2755] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:41:56.362442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4f799-5600-4a3d-a52e-27d7bcce7e29 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:56.372946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4f799-5600-4a3d-a52e-27d7bcce7e29 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:56.377616 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d6eaf-b05e-47ef-8659-f6f23ff5c814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:56.378277 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d6eaf-b05e-47ef-8659-f6f23ff5c814 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:56.378565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d6eaf-b05e-47ef-8659-f6f23ff5c814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:56.378846 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d6eaf-b05e-47ef-8659-f6f23ff5c814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:56.379572 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d6eaf-b05e-47ef-8659-f6f23ff5c814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:56.381172 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d6eaf-b05e-47ef-8659-f6f23ff5c814 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:56.392179 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1386/2756] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:41:56.396282 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8939b2-2d90-4582-af80-4ee06100cf7e admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:56.406210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8939b2-2d90-4582-af80-4ee06100cf7e admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:56.411065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396cda3-1929-48d0-9eb1-734463b39d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:56.411931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396cda3-1929-48d0-9eb1-734463b39d10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:56.412214 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396cda3-1929-48d0-9eb1-734463b39d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:56.412488 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396cda3-1929-48d0-9eb1-734463b39d10 admin admin] PATH_INFO: `/v3/projects/751699db1dfb4f38856f4d07a74f92a1/users/a3475b3ec4704d5e9d5a350241de6c24/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:56.428245 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396cda3-1929-48d0-9eb1-734463b39d10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=751699db1dfb4f38856f4d07a74f92a1, user_id=a3475b3ec4704d5e9d5a350241de6c24, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:56.431538 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396cda3-1929-48d0-9eb1-734463b39d10 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:56.444112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1371/2757] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:41:56 2022] PUT /identity/v3/projects/751699db1dfb4f38856f4d07a74f92a1/users/a3475b3ec4704d5e9d5a350241de6c24/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:41:56.456135 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdc5e1-cc2d-4237-83e4-a09c7673edcc None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:56.456436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdc5e1-cc2d-4237-83e4-a09c7673edcc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:56.456725 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdc5e1-cc2d-4237-83e4-a09c7673edcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:56.483645 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-f2bdc5e1-cc2d-4237-83e4-a09c7673edcc None None] MFA Rules not processed for user `a3475b3ec4704d5e9d5a350241de6c24`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:41:56.484905 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-f2bdc5e1-cc2d-4237-83e4-a09c7673edcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:41:56.549651 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1387/2758] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:41:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:41:56.554258 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396cda3-1929-48d0-9eb1-734463b39d10 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:56.616448 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396cda3-1929-48d0-9eb1-734463b39d10 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:56.622282 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8752c05-4b12-487b-b468-7a6fd579b401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:56.622946 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8752c05-4b12-487b-b468-7a6fd579b401 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:56.623236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8752c05-4b12-487b-b468-7a6fd579b401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:56.623517 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8752c05-4b12-487b-b468-7a6fd579b401 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:56.624220 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8752c05-4b12-487b-b468-7a6fd579b401 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:56.626184 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8752c05-4b12-487b-b468-7a6fd579b401 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:56.652408 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1372/2759] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:41:56 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:41:56.656387 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bdc5e1-cc2d-4237-83e4-a09c7673edcc None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:56.668380 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bdc5e1-cc2d-4237-83e4-a09c7673edcc None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:56.687170 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:41:56.728320 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e67adf-7d2c-45ab-b296-75955db1406f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:56.729006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e67adf-7d2c-45ab-b296-75955db1406f admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:56.729321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e67adf-7d2c-45ab-b296-75955db1406f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:56.729632 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e67adf-7d2c-45ab-b296-75955db1406f admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:56.730452 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e67adf-7d2c-45ab-b296-75955db1406f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:56.732385 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e67adf-7d2c-45ab-b296-75955db1406f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:56.755565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1388/2760] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:41:56 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:41:56.756919 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8752c05-4b12-487b-b468-7a6fd579b401 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:56.767379 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8752c05-4b12-487b-b468-7a6fd579b401 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:56.772049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8b5f8-2b36-43f8-ae15-b2100b2cd18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:56.772710 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b5f8-2b36-43f8-ae15-b2100b2cd18f admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:56.772997 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b5f8-2b36-43f8-ae15-b2100b2cd18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:56.773298 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b5f8-2b36-43f8-ae15-b2100b2cd18f admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:56.774089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8b5f8-2b36-43f8-ae15-b2100b2cd18f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:56.775778 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8b5f8-2b36-43f8-ae15-b2100b2cd18f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:56.786640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1373/2761] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:41:56.791263 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e67adf-7d2c-45ab-b296-75955db1406f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:56.801128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e67adf-7d2c-45ab-b296-75955db1406f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:56.806373 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e361415-a7ff-4272-b13d-d54b56857deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:56.807250 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e361415-a7ff-4272-b13d-d54b56857deb admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:56.807616 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e361415-a7ff-4272-b13d-d54b56857deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:56.807901 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e361415-a7ff-4272-b13d-d54b56857deb admin admin] PATH_INFO: `/v3/projects/062c472274ba4819915a398462653281/users/e791f2eef00b418890633842a5913842/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:56.822550 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e361415-a7ff-4272-b13d-d54b56857deb admin admin] RBAC: Authorizing `identity:create_grant(project_id=062c472274ba4819915a398462653281, user_id=e791f2eef00b418890633842a5913842, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:56.825204 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e361415-a7ff-4272-b13d-d54b56857deb admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:56.837768 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1389/2762] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:41:56 2022] PUT /identity/v3/projects/062c472274ba4819915a398462653281/users/e791f2eef00b418890633842a5913842/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:41:56.841673 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8b5f8-2b36-43f8-ae15-b2100b2cd18f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:56.851671 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8b5f8-2b36-43f8-ae15-b2100b2cd18f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:56.908177 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5803e6d-7850-4b54-bb5f-71640a5799c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:56.908850 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5803e6d-7850-4b54-bb5f-71640a5799c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:56.909129 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5803e6d-7850-4b54-bb5f-71640a5799c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:56.909441 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5803e6d-7850-4b54-bb5f-71640a5799c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:56.910159 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5803e6d-7850-4b54-bb5f-71640a5799c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:56.911759 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5803e6d-7850-4b54-bb5f-71640a5799c5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:56.922514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1374/2763] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:41:56.926940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e361415-a7ff-4272-b13d-d54b56857deb admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:56.937488 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e361415-a7ff-4272-b13d-d54b56857deb admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:56.942619 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8f548-c162-427c-82fa-58043633f51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:56.943676 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8f548-c162-427c-82fa-58043633f51d admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:56.944009 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8f548-c162-427c-82fa-58043633f51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:56.944392 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8f548-c162-427c-82fa-58043633f51d admin admin] PATH_INFO: `/v3/projects/062c472274ba4819915a398462653281/users/e791f2eef00b418890633842a5913842/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:56.947293 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8f548-c162-427c-82fa-58043633f51d admin admin] RBAC: Authorizing `identity:create_grant(project_id=062c472274ba4819915a398462653281, user_id=e791f2eef00b418890633842a5913842, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:56.950177 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8f548-c162-427c-82fa-58043633f51d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:56.961349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1390/2764] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:41:56 2022] PUT /identity/v3/projects/062c472274ba4819915a398462653281/users/e791f2eef00b418890633842a5913842/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:41:56.971929 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e6383-0a63-4686-92ca-9227fa7925d0 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:56.972223 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e6383-0a63-4686-92ca-9227fa7925d0 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:56.972526 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e6383-0a63-4686-92ca-9227fa7925d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:56.999659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-7a0e6383-0a63-4686-92ca-9227fa7925d0 None None] MFA Rules not processed for user `e791f2eef00b418890633842a5913842`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:41:57.000865 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7a0e6383-0a63-4686-92ca-9227fa7925d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:41:57.018921 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:41:57.070236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1375/2765] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:41:56 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:41:57.081454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8f548-c162-427c-82fa-58043633f51d admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:57.083177 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8de8f548-c162-427c-82fa-58043633f51d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:41:57.140255 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8f548-c162-427c-82fa-58043633f51d admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:57.143893 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008eb195-dd81-4226-af45-c12b1e405a51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:57.144588 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eb195-dd81-4226-af45-c12b1e405a51 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:57.144868 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eb195-dd81-4226-af45-c12b1e405a51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:57.145143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eb195-dd81-4226-af45-c12b1e405a51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:57.154614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008eb195-dd81-4226-af45-c12b1e405a51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:57.156047 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008eb195-dd81-4226-af45-c12b1e405a51 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:57.166515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1391/2766] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:41:57 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:41:57.239743 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e6383-0a63-4686-92ca-9227fa7925d0 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:41:57.249607 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e6383-0a63-4686-92ca-9227fa7925d0 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:41:57.253275 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b111cca-e270-4ae0-9599-57505a3c4e23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:41:57.253933 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b111cca-e270-4ae0-9599-57505a3c4e23 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:41:57.254221 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b111cca-e270-4ae0-9599-57505a3c4e23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:41:57.254503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b111cca-e270-4ae0-9599-57505a3c4e23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:41:57.311836 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b111cca-e270-4ae0-9599-57505a3c4e23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:41:57.313347 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b111cca-e270-4ae0-9599-57505a3c4e23 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:41:57.323009 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1376/2767] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:41:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:42:07.161485 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=60,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:07.318307 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=48 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:35.685698 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008eb195-dd81-4226-af45-c12b1e405a51 service cinder] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:35.773580 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008eb195-dd81-4226-af45-c12b1e405a51 service cinder] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:35.778604 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b7bd3-a24b-49c9-b13e-af31a57149e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:35.779465 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b7bd3-a24b-49c9-b13e-af31a57149e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:35.779768 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b7bd3-a24b-49c9-b13e-af31a57149e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:35.780464 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b7bd3-a24b-49c9-b13e-af31a57149e2 admin admin] PATH_INFO: `/v3/users/a3475b3ec4704d5e9d5a350241de6c24` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:35.781227 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b7bd3-a24b-49c9-b13e-af31a57149e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3475b3ec4704d5e9d5a350241de6c24)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:35.783418 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b7bd3-a24b-49c9-b13e-af31a57149e2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:35.830822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-887b7bd3-a24b-49c9-b13e-af31a57149e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3475b3ec4704d5e9d5a350241de6c24'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:35.831400 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-887b7bd3-a24b-49c9-b13e-af31a57149e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3475b3ec4704d5e9d5a350241de6c24'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:35.843227 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-887b7bd3-a24b-49c9-b13e-af31a57149e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3475b3ec4704d5e9d5a350241de6c24'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:35.843860 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-887b7bd3-a24b-49c9-b13e-af31a57149e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3475b3ec4704d5e9d5a350241de6c24'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:35.851412 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-887b7bd3-a24b-49c9-b13e-af31a57149e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3475b3ec4704d5e9d5a350241de6c24'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:35.863899 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1392/2768] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:42:35 2022] DELETE /identity/v3/users/a3475b3ec4704d5e9d5a350241de6c24 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:35.868917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46858c52-e5e4-44a8-be0d-748cf0c579e4 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:35.869452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46858c52-e5e4-44a8-be0d-748cf0c579e4 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:35.869556 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46858c52-e5e4-44a8-be0d-748cf0c579e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:35.917875 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-46858c52-e5e4-44a8-be0d-748cf0c579e4 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:35.919081 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-46858c52-e5e4-44a8-be0d-748cf0c579e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:35.977759 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1377/2769] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:35.988008 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b7bd3-a24b-49c9-b13e-af31a57149e2 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:35.989515 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-887b7bd3-a24b-49c9-b13e-af31a57149e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:36.050034 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:36.084123 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b7bd3-a24b-49c9-b13e-af31a57149e2 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:36.087719 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874690d6-c085-44e0-8b88-2219044db093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:36.088389 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874690d6-c085-44e0-8b88-2219044db093 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:36.088670 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874690d6-c085-44e0-8b88-2219044db093 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:36.088970 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874690d6-c085-44e0-8b88-2219044db093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:36.098562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874690d6-c085-44e0-8b88-2219044db093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:36.099995 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874690d6-c085-44e0-8b88-2219044db093 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:36.110100 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1393/2770] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:42:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:42:36.386767 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46858c52-e5e4-44a8-be0d-748cf0c579e4 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:36.388217 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-46858c52-e5e4-44a8-be0d-748cf0c579e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:36.400010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46858c52-e5e4-44a8-be0d-748cf0c579e4 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:36.405685 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770e9cc-0afc-4bc7-b006-01a162b091f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:36.406912 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770e9cc-0afc-4bc7-b006-01a162b091f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:36.407255 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770e9cc-0afc-4bc7-b006-01a162b091f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:36.407537 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770e9cc-0afc-4bc7-b006-01a162b091f5 admin admin] PATH_INFO: `/v3/projects/751699db1dfb4f38856f4d07a74f92a1` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:36.409029 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770e9cc-0afc-4bc7-b006-01a162b091f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=751699db1dfb4f38856f4d07a74f92a1)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:36.410996 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770e9cc-0afc-4bc7-b006-01a162b091f5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:36.460182 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5770e9cc-0afc-4bc7-b006-01a162b091f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '751699db1dfb4f38856f4d07a74f92a1', 'request_id': 'req-5770e9cc-0afc-4bc7-b006-01a162b091f5'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:36.468575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5770e9cc-0afc-4bc7-b006-01a162b091f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '751699db1dfb4f38856f4d07a74f92a1', 'request_id': 'req-5770e9cc-0afc-4bc7-b006-01a162b091f5'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:36.474170 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5770e9cc-0afc-4bc7-b006-01a162b091f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '751699db1dfb4f38856f4d07a74f92a1', 'request_id': 'req-5770e9cc-0afc-4bc7-b006-01a162b091f5'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:36.486442 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5770e9cc-0afc-4bc7-b006-01a162b091f5 admin admin] The token cache is being invalidate because project 751699db1dfb4f38856f4d07a74f92a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:42:36.486735 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5770e9cc-0afc-4bc7-b006-01a162b091f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:36.487380 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-5770e9cc-0afc-4bc7-b006-01a162b091f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:36.488991 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1378/2771] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:42:36 2022] DELETE /identity/v3/projects/751699db1dfb4f38856f4d07a74f92a1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:36.493015 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874690d6-c085-44e0-8b88-2219044db093 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:36.502730 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874690d6-c085-44e0-8b88-2219044db093 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:36.559904 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2774fb-8e51-4147-996b-9a1cefe2daaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:36.560785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2774fb-8e51-4147-996b-9a1cefe2daaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:36.561070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2774fb-8e51-4147-996b-9a1cefe2daaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:36.561380 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2774fb-8e51-4147-996b-9a1cefe2daaf admin admin] PATH_INFO: `/v3/users/e791f2eef00b418890633842a5913842` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:36.562590 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2774fb-8e51-4147-996b-9a1cefe2daaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e791f2eef00b418890633842a5913842)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:36.564544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2774fb-8e51-4147-996b-9a1cefe2daaf admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:36.610099 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9d2774fb-8e51-4147-996b-9a1cefe2daaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e791f2eef00b418890633842a5913842'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:36.610671 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9d2774fb-8e51-4147-996b-9a1cefe2daaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e791f2eef00b418890633842a5913842'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:36.622591 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9d2774fb-8e51-4147-996b-9a1cefe2daaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e791f2eef00b418890633842a5913842'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:36.623233 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9d2774fb-8e51-4147-996b-9a1cefe2daaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e791f2eef00b418890633842a5913842'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:36.631499 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-9d2774fb-8e51-4147-996b-9a1cefe2daaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e791f2eef00b418890633842a5913842'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:36.645109 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1394/2772] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:42:36 2022] DELETE /identity/v3/users/e791f2eef00b418890633842a5913842 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:36.912490 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770e9cc-0afc-4bc7-b006-01a162b091f5 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:36.914027 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-5770e9cc-0afc-4bc7-b006-01a162b091f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:36.928350 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770e9cc-0afc-4bc7-b006-01a162b091f5 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:36.985049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:36.991780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7db2b-eb0d-40a5-920a-06ca361e9720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:36.992658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7db2b-eb0d-40a5-920a-06ca361e9720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:36.993119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7db2b-eb0d-40a5-920a-06ca361e9720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:36.993486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7db2b-eb0d-40a5-920a-06ca361e9720 admin admin] PATH_INFO: `/v3/projects/062c472274ba4819915a398462653281` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:36.994884 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7db2b-eb0d-40a5-920a-06ca361e9720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=062c472274ba4819915a398462653281)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:36.997776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7db2b-eb0d-40a5-920a-06ca361e9720 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:37.043826 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-94d7db2b-eb0d-40a5-920a-06ca361e9720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '062c472274ba4819915a398462653281', 'request_id': 'req-94d7db2b-eb0d-40a5-920a-06ca361e9720'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:37.051146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-94d7db2b-eb0d-40a5-920a-06ca361e9720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '062c472274ba4819915a398462653281', 'request_id': 'req-94d7db2b-eb0d-40a5-920a-06ca361e9720'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:37.056708 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-94d7db2b-eb0d-40a5-920a-06ca361e9720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '062c472274ba4819915a398462653281', 'request_id': 'req-94d7db2b-eb0d-40a5-920a-06ca361e9720'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:37.068601 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-94d7db2b-eb0d-40a5-920a-06ca361e9720 admin admin] The token cache is being invalidate because project 062c472274ba4819915a398462653281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:42:37.068887 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-94d7db2b-eb0d-40a5-920a-06ca361e9720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:37.069555 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-94d7db2b-eb0d-40a5-920a-06ca361e9720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:37.071103 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1379/2773] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:42:36 2022] DELETE /identity/v3/projects/062c472274ba4819915a398462653281 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:37.077150 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9d9b5-e0b5-473f-819c-4aa314734a9d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:37.077680 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9d9b5-e0b5-473f-819c-4aa314734a9d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:37.077790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9d9b5-e0b5-473f-819c-4aa314734a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:37.106430 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-18f9d9b5-e0b5-473f-819c-4aa314734a9d None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:37.107672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-18f9d9b5-e0b5-473f-819c-4aa314734a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:37.138452 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:37.167874 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1395/2774] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:37.176834 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819ac1f-c39a-46bc-9ea8-d01509b3579c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:37.177126 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819ac1f-c39a-46bc-9ea8-d01509b3579c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:37.177444 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819ac1f-c39a-46bc-9ea8-d01509b3579c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:37.199182 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-b819ac1f-c39a-46bc-9ea8-d01509b3579c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:37.200395 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b819ac1f-c39a-46bc-9ea8-d01509b3579c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:37.208319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1380/2775] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:37.212589 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9d9b5-e0b5-473f-819c-4aa314734a9d None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:37.222297 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9d9b5-e0b5-473f-819c-4aa314734a9d None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:37.226854 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b64aa-9c14-42cb-b06b-a3034f4eaffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:37.227508 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b64aa-9c14-42cb-b06b-a3034f4eaffd admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:37.227795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b64aa-9c14-42cb-b06b-a3034f4eaffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:37.228075 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b64aa-9c14-42cb-b06b-a3034f4eaffd admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:37.228804 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b64aa-9c14-42cb-b06b-a3034f4eaffd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:42:37.229107 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b64aa-9c14-42cb-b06b-a3034f4eaffd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:37.230775 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b64aa-9c14-42cb-b06b-a3034f4eaffd admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:37.242196 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1396/2776] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:42:37.247176 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819ac1f-c39a-46bc-9ea8-d01509b3579c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:37.256334 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819ac1f-c39a-46bc-9ea8-d01509b3579c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:37.261630 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e459fe-2e69-4fbc-88a7-3ccd60b63c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:37.262277 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e459fe-2e69-4fbc-88a7-3ccd60b63c81 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:37.262560 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e459fe-2e69-4fbc-88a7-3ccd60b63c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:37.262838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e459fe-2e69-4fbc-88a7-3ccd60b63c81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:37.263548 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e459fe-2e69-4fbc-88a7-3ccd60b63c81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:37.265489 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e459fe-2e69-4fbc-88a7-3ccd60b63c81 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:37.292493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1381/2777] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:42:37 2022] POST /identity/v3/projects => generated 387 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:42:37.294348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b64aa-9c14-42cb-b06b-a3034f4eaffd admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:37.304504 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b64aa-9c14-42cb-b06b-a3034f4eaffd admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:37.361041 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3cfa9-80fe-4571-a149-6468486588c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:37.361703 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3cfa9-80fe-4571-a149-6468486588c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:37.361993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3cfa9-80fe-4571-a149-6468486588c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:37.362276 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3cfa9-80fe-4571-a149-6468486588c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:37.362982 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3cfa9-80fe-4571-a149-6468486588c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:37.364902 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3cfa9-80fe-4571-a149-6468486588c8 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:37.384866 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1397/2778] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:42:37 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:42:37.388939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e459fe-2e69-4fbc-88a7-3ccd60b63c81 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:37.399413 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e459fe-2e69-4fbc-88a7-3ccd60b63c81 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:37.404089 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eaf4a1-7097-467d-9377-0d277fa14066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:37.404742 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaf4a1-7097-467d-9377-0d277fa14066 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:37.405026 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaf4a1-7097-467d-9377-0d277fa14066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:37.405330 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaf4a1-7097-467d-9377-0d277fa14066 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:37.406049 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eaf4a1-7097-467d-9377-0d277fa14066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:37.407668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eaf4a1-7097-467d-9377-0d277fa14066 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:37.419547 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1382/2779] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:42:37.423381 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3cfa9-80fe-4571-a149-6468486588c8 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:37.434473 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3cfa9-80fe-4571-a149-6468486588c8 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:37.440030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526f237-d70b-43a8-abc3-fcd2ef589b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:37.440698 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526f237-d70b-43a8-abc3-fcd2ef589b6b admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:37.440989 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526f237-d70b-43a8-abc3-fcd2ef589b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:37.441269 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526f237-d70b-43a8-abc3-fcd2ef589b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:37.442048 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526f237-d70b-43a8-abc3-fcd2ef589b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:37.443636 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526f237-d70b-43a8-abc3-fcd2ef589b6b admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:37.454162 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1398/2780] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:42:37.458465 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eaf4a1-7097-467d-9377-0d277fa14066 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:37.468698 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eaf4a1-7097-467d-9377-0d277fa14066 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:37.474477 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925ed07-a6ed-4a82-bb09-41771ff5b252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:37.475358 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925ed07-a6ed-4a82-bb09-41771ff5b252 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:37.475656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925ed07-a6ed-4a82-bb09-41771ff5b252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:37.475910 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925ed07-a6ed-4a82-bb09-41771ff5b252 admin admin] PATH_INFO: `/v3/projects/13f0f08aa9d44a3482ae4f52d3656f43/users/2d68efc329b34856ba80399bb5f3a023/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:37.496276 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925ed07-a6ed-4a82-bb09-41771ff5b252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f0f08aa9d44a3482ae4f52d3656f43, user_id=2d68efc329b34856ba80399bb5f3a023, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:37.498999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925ed07-a6ed-4a82-bb09-41771ff5b252 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:37.516786 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1383/2781] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:42:37 2022] PUT /identity/v3/projects/13f0f08aa9d44a3482ae4f52d3656f43/users/2d68efc329b34856ba80399bb5f3a023/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:42:37.527032 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea5a09-a2d1-4b87-8230-a7e497b2db55 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:37.527409 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea5a09-a2d1-4b87-8230-a7e497b2db55 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:37.527696 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea5a09-a2d1-4b87-8230-a7e497b2db55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:37.555586 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-b1ea5a09-a2d1-4b87-8230-a7e497b2db55 None None] MFA Rules not processed for user `2d68efc329b34856ba80399bb5f3a023`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:37.556790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b1ea5a09-a2d1-4b87-8230-a7e497b2db55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:37.621659 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1399/2782] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:37.626124 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925ed07-a6ed-4a82-bb09-41771ff5b252 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:37.691420 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925ed07-a6ed-4a82-bb09-41771ff5b252 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:37.696079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830958f-a7c4-4c07-9909-1f786d391b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:37.696744 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830958f-a7c4-4c07-9909-1f786d391b3f admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:37.697034 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830958f-a7c4-4c07-9909-1f786d391b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:37.697341 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830958f-a7c4-4c07-9909-1f786d391b3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:37.698046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830958f-a7c4-4c07-9909-1f786d391b3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:37.700079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830958f-a7c4-4c07-9909-1f786d391b3f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:37.725797 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1384/2783] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:42:37 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:42:37.730523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea5a09-a2d1-4b87-8230-a7e497b2db55 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:37.744328 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea5a09-a2d1-4b87-8230-a7e497b2db55 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:37.782988 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:37.813412 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a955288-97e9-4c6e-aebb-a95a377caf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:37.814113 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a955288-97e9-4c6e-aebb-a95a377caf8a admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:37.814404 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a955288-97e9-4c6e-aebb-a95a377caf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:37.814684 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a955288-97e9-4c6e-aebb-a95a377caf8a admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:37.815399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a955288-97e9-4c6e-aebb-a95a377caf8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:37.817379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a955288-97e9-4c6e-aebb-a95a377caf8a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:37.842335 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1400/2784] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:42:37 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:42:37.845560 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830958f-a7c4-4c07-9909-1f786d391b3f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:37.856487 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830958f-a7c4-4c07-9909-1f786d391b3f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:37.861103 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337be8cf-0edc-423c-9617-067c0687f233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:37.861791 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337be8cf-0edc-423c-9617-067c0687f233 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:37.862078 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337be8cf-0edc-423c-9617-067c0687f233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:37.862354 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337be8cf-0edc-423c-9617-067c0687f233 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:37.863073 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337be8cf-0edc-423c-9617-067c0687f233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:37.864660 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337be8cf-0edc-423c-9617-067c0687f233 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:37.866830 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:37.876876 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1385/2785] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:42:37.881609 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a955288-97e9-4c6e-aebb-a95a377caf8a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:37.891604 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a955288-97e9-4c6e-aebb-a95a377caf8a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:37.896224 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa8152-be52-41fa-ac73-0bbc92a63b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:37.897196 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa8152-be52-41fa-ac73-0bbc92a63b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:37.897518 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa8152-be52-41fa-ac73-0bbc92a63b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:37.897803 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa8152-be52-41fa-ac73-0bbc92a63b73 admin admin] PATH_INFO: `/v3/projects/c170072ee0c54f53a49b5e230aed9c83/users/ac79fd082add4ae28dafba975d047982/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:37.914548 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa8152-be52-41fa-ac73-0bbc92a63b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c170072ee0c54f53a49b5e230aed9c83, user_id=ac79fd082add4ae28dafba975d047982, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:37.917923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa8152-be52-41fa-ac73-0bbc92a63b73 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:37.930812 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1401/2786] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:42:37 2022] PUT /identity/v3/projects/c170072ee0c54f53a49b5e230aed9c83/users/ac79fd082add4ae28dafba975d047982/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:37.935146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337be8cf-0edc-423c-9617-067c0687f233 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:37.945062 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337be8cf-0edc-423c-9617-067c0687f233 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:38.001274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c50f62-fbc9-43fd-bd2c-d65b238289b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:38.001981 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c50f62-fbc9-43fd-bd2c-d65b238289b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:38.002261 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c50f62-fbc9-43fd-bd2c-d65b238289b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:38.002540 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c50f62-fbc9-43fd-bd2c-d65b238289b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:38.003257 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c50f62-fbc9-43fd-bd2c-d65b238289b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:38.004843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c50f62-fbc9-43fd-bd2c-d65b238289b5 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:38.014581 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1386/2787] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:42:38.018837 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa8152-be52-41fa-ac73-0bbc92a63b73 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:38.029851 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa8152-be52-41fa-ac73-0bbc92a63b73 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:38.034808 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a669d-d7d2-438a-b4fc-227f07db4a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:38.035682 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a669d-d7d2-438a-b4fc-227f07db4a31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:38.035966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a669d-d7d2-438a-b4fc-227f07db4a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:38.036244 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a669d-d7d2-438a-b4fc-227f07db4a31 admin admin] PATH_INFO: `/v3/projects/c170072ee0c54f53a49b5e230aed9c83/users/ac79fd082add4ae28dafba975d047982/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:38.038249 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a669d-d7d2-438a-b4fc-227f07db4a31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c170072ee0c54f53a49b5e230aed9c83, user_id=ac79fd082add4ae28dafba975d047982, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:38.040862 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a669d-d7d2-438a-b4fc-227f07db4a31 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:38.051728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1402/2788] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:42:38 2022] PUT /identity/v3/projects/c170072ee0c54f53a49b5e230aed9c83/users/ac79fd082add4ae28dafba975d047982/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:38.062894 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd52e91-0d3a-4f3a-813d-f86e4a9c40ab None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:38.063189 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd52e91-0d3a-4f3a-813d-f86e4a9c40ab None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:38.063504 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd52e91-0d3a-4f3a-813d-f86e4a9c40ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:38.090668 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-ddd52e91-0d3a-4f3a-813d-f86e4a9c40ab None None] MFA Rules not processed for user `ac79fd082add4ae28dafba975d047982`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:38.091871 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-ddd52e91-0d3a-4f3a-813d-f86e4a9c40ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:38.166003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1387/2789] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:42:38.172926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a669d-d7d2-438a-b4fc-227f07db4a31 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:38.174777 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-7c0a669d-d7d2-438a-b4fc-227f07db4a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:38.234986 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a669d-d7d2-438a-b4fc-227f07db4a31 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:38.238654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828d00f-3cf7-4519-805d-a51360811620 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:38.239296 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828d00f-3cf7-4519-805d-a51360811620 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:38.239579 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828d00f-3cf7-4519-805d-a51360811620 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:38.239857 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828d00f-3cf7-4519-805d-a51360811620 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:38.248894 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828d00f-3cf7-4519-805d-a51360811620 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:38.250401 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828d00f-3cf7-4519-805d-a51360811620 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:38.259937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1403/2790] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:42:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:42:38.790372 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd52e91-0d3a-4f3a-813d-f86e4a9c40ab None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:38.855242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd52e91-0d3a-4f3a-813d-f86e4a9c40ab None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:38.860795 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef63b63-43a5-48df-ba6d-5e6969934cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:38.861699 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef63b63-43a5-48df-ba6d-5e6969934cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:38.861990 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef63b63-43a5-48df-ba6d-5e6969934cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:38.862274 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef63b63-43a5-48df-ba6d-5e6969934cb3 admin admin] PATH_INFO: `/v3/users/2d68efc329b34856ba80399bb5f3a023` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:38.863467 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef63b63-43a5-48df-ba6d-5e6969934cb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d68efc329b34856ba80399bb5f3a023)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:38.865430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef63b63-43a5-48df-ba6d-5e6969934cb3 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:38.887917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:38.914330 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2ef63b63-43a5-48df-ba6d-5e6969934cb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d68efc329b34856ba80399bb5f3a023'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:38.926297 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2ef63b63-43a5-48df-ba6d-5e6969934cb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d68efc329b34856ba80399bb5f3a023'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:38.934541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2ef63b63-43a5-48df-ba6d-5e6969934cb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d68efc329b34856ba80399bb5f3a023'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:38.935190 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2ef63b63-43a5-48df-ba6d-5e6969934cb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d68efc329b34856ba80399bb5f3a023'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:38.936031 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-2ef63b63-43a5-48df-ba6d-5e6969934cb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d68efc329b34856ba80399bb5f3a023'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:38.948837 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1388/2791] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:42:38 2022] DELETE /identity/v3/users/2d68efc329b34856ba80399bb5f3a023 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:38.953523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac916bdd-6187-4ec4-a77f-2b7c53a86fcb None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:38.953822 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac916bdd-6187-4ec4-a77f-2b7c53a86fcb None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:38.954105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac916bdd-6187-4ec4-a77f-2b7c53a86fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:38.972265 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ac916bdd-6187-4ec4-a77f-2b7c53a86fcb None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:38.973501 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ac916bdd-6187-4ec4-a77f-2b7c53a86fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:39.032133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1404/2792] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:39.041793 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef63b63-43a5-48df-ba6d-5e6969934cb3 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:39.043486 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-2ef63b63-43a5-48df-ba6d-5e6969934cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:39.101112 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef63b63-43a5-48df-ba6d-5e6969934cb3 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:39.104776 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba51e5-40db-4221-84b6-8710d5b4b1ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:39.105547 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba51e5-40db-4221-84b6-8710d5b4b1ff service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:39.105820 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba51e5-40db-4221-84b6-8710d5b4b1ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:39.106096 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba51e5-40db-4221-84b6-8710d5b4b1ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:39.115184 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba51e5-40db-4221-84b6-8710d5b4b1ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:39.116618 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba51e5-40db-4221-84b6-8710d5b4b1ff service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:39.128961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1389/2793] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:42:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:42:39.398307 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac916bdd-6187-4ec4-a77f-2b7c53a86fcb None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:39.399945 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ac916bdd-6187-4ec4-a77f-2b7c53a86fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:39.414850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac916bdd-6187-4ec4-a77f-2b7c53a86fcb None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:39.420519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:39.421412 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:39.421697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:39.421974 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99 admin admin] PATH_INFO: `/v3/projects/13f0f08aa9d44a3482ae4f52d3656f43` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:39.423179 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f0f08aa9d44a3482ae4f52d3656f43)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:39.425044 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:39.449572 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:39.474486 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f0f08aa9d44a3482ae4f52d3656f43', 'request_id': 'req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:39.480592 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f0f08aa9d44a3482ae4f52d3656f43', 'request_id': 'req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:39.489428 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f0f08aa9d44a3482ae4f52d3656f43', 'request_id': 'req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:39.501876 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99 admin admin] The token cache is being invalidate because project 13f0f08aa9d44a3482ae4f52d3656f43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:42:39.502176 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:39.502826 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:39.504379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1405/2794] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:42:39 2022] DELETE /identity/v3/projects/13f0f08aa9d44a3482ae4f52d3656f43 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:39.508483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba51e5-40db-4221-84b6-8710d5b4b1ff service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:39.519243 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba51e5-40db-4221-84b6-8710d5b4b1ff service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:39.577635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd536786-feca-4561-b7f3-b87064065e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:39.578422 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd536786-feca-4561-b7f3-b87064065e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:39.578706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd536786-feca-4561-b7f3-b87064065e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:39.578979 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd536786-feca-4561-b7f3-b87064065e06 admin admin] PATH_INFO: `/v3/users/ac79fd082add4ae28dafba975d047982` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:39.580161 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd536786-feca-4561-b7f3-b87064065e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac79fd082add4ae28dafba975d047982)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:39.582325 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd536786-feca-4561-b7f3-b87064065e06 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:39.626924 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:39.629787 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bd536786-feca-4561-b7f3-b87064065e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac79fd082add4ae28dafba975d047982'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:39.643700 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bd536786-feca-4561-b7f3-b87064065e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac79fd082add4ae28dafba975d047982'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:39.651907 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bd536786-feca-4561-b7f3-b87064065e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac79fd082add4ae28dafba975d047982'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:39.652541 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bd536786-feca-4561-b7f3-b87064065e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac79fd082add4ae28dafba975d047982'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:39.653146 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-bd536786-feca-4561-b7f3-b87064065e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac79fd082add4ae28dafba975d047982'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:39.665715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1390/2795] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:42:39 2022] DELETE /identity/v3/users/ac79fd082add4ae28dafba975d047982 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:39.944724 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:39.946323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:39.960946 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd4fdc-d1b6-40f3-95af-a1ef314ccf99 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:40.023267 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ecd51-7a7c-4f59-b89f-7f2af0d99ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:40.024137 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ecd51-7a7c-4f59-b89f-7f2af0d99ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:40.024426 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ecd51-7a7c-4f59-b89f-7f2af0d99ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:40.024707 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ecd51-7a7c-4f59-b89f-7f2af0d99ea6 admin admin] PATH_INFO: `/v3/projects/c170072ee0c54f53a49b5e230aed9c83` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:40.025924 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ecd51-7a7c-4f59-b89f-7f2af0d99ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c170072ee0c54f53a49b5e230aed9c83)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:40.028442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ecd51-7a7c-4f59-b89f-7f2af0d99ea6 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:40.076624 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-587ecd51-7a7c-4f59-b89f-7f2af0d99ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c170072ee0c54f53a49b5e230aed9c83', 'request_id': 'req-587ecd51-7a7c-4f59-b89f-7f2af0d99ea6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:40.082321 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-587ecd51-7a7c-4f59-b89f-7f2af0d99ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c170072ee0c54f53a49b5e230aed9c83', 'request_id': 'req-587ecd51-7a7c-4f59-b89f-7f2af0d99ea6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:40.090533 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-587ecd51-7a7c-4f59-b89f-7f2af0d99ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c170072ee0c54f53a49b5e230aed9c83', 'request_id': 'req-587ecd51-7a7c-4f59-b89f-7f2af0d99ea6'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:40.103338 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-587ecd51-7a7c-4f59-b89f-7f2af0d99ea6 admin admin] The token cache is being invalidate because project c170072ee0c54f53a49b5e230aed9c83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:42:40.103629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-587ecd51-7a7c-4f59-b89f-7f2af0d99ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:40.104265 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-587ecd51-7a7c-4f59-b89f-7f2af0d99ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:40.106070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1406/2796] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:42:39 2022] DELETE /identity/v3/projects/c170072ee0c54f53a49b5e230aed9c83 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:42:40.111298 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc772eee-944d-4199-90eb-c3960e8cc68b None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:40.111780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc772eee-944d-4199-90eb-c3960e8cc68b None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:40.111899 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc772eee-944d-4199-90eb-c3960e8cc68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:40.130615 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-dc772eee-944d-4199-90eb-c3960e8cc68b None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:40.131779 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-dc772eee-944d-4199-90eb-c3960e8cc68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:40.194294 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1391/2797] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:40.202803 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0bdfb-8efd-4cb6-bfdf-43920f4191e6 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:40.203196 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0bdfb-8efd-4cb6-bfdf-43920f4191e6 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:40.203402 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0bdfb-8efd-4cb6-bfdf-43920f4191e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:40.224423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-e4b0bdfb-8efd-4cb6-bfdf-43920f4191e6 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:40.225666 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-e4b0bdfb-8efd-4cb6-bfdf-43920f4191e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:40.233794 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1407/2798] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:40.238121 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc772eee-944d-4199-90eb-c3960e8cc68b None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:40.248857 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc772eee-944d-4199-90eb-c3960e8cc68b None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:40.253967 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1784d7-94c4-4e80-9e43-e98d7df7f0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:40.254715 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1784d7-94c4-4e80-9e43-e98d7df7f0bc admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:40.255019 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1784d7-94c4-4e80-9e43-e98d7df7f0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:40.255308 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1784d7-94c4-4e80-9e43-e98d7df7f0bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:40.256042 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1784d7-94c4-4e80-9e43-e98d7df7f0bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:42:40.256359 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1784d7-94c4-4e80-9e43-e98d7df7f0bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:40.258053 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1784d7-94c4-4e80-9e43-e98d7df7f0bc admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:40.271075 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1392/2799] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:42:40.275665 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0bdfb-8efd-4cb6-bfdf-43920f4191e6 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:40.285350 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0bdfb-8efd-4cb6-bfdf-43920f4191e6 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:40.290178 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2a175-bd5e-4139-9092-18299d843ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:40.290828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2a175-bd5e-4139-9092-18299d843ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:40.291111 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2a175-bd5e-4139-9092-18299d843ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:40.291392 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2a175-bd5e-4139-9092-18299d843ec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:40.292078 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2a175-bd5e-4139-9092-18299d843ec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:40.294220 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2a175-bd5e-4139-9092-18299d843ec3 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:40.322658 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1408/2800] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:42:40 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:42:40.325163 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1784d7-94c4-4e80-9e43-e98d7df7f0bc admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:40.335585 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1784d7-94c4-4e80-9e43-e98d7df7f0bc admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:40.394169 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed5b2e-0268-477c-a5cc-bb6735f85597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:40.394852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed5b2e-0268-477c-a5cc-bb6735f85597 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:40.395142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed5b2e-0268-477c-a5cc-bb6735f85597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:40.395425 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed5b2e-0268-477c-a5cc-bb6735f85597 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:40.396111 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed5b2e-0268-477c-a5cc-bb6735f85597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:40.398078 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed5b2e-0268-477c-a5cc-bb6735f85597 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:40.418380 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1393/2801] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:42:40 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:42:40.422861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2a175-bd5e-4139-9092-18299d843ec3 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:40.433306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2a175-bd5e-4139-9092-18299d843ec3 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:40.438082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b8280-51a4-4257-b2b7-4368e315f1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:40.438821 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b8280-51a4-4257-b2b7-4368e315f1d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:40.439174 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b8280-51a4-4257-b2b7-4368e315f1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:40.439518 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b8280-51a4-4257-b2b7-4368e315f1d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:40.440369 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b8280-51a4-4257-b2b7-4368e315f1d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:40.442075 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b8280-51a4-4257-b2b7-4368e315f1d3 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:40.447521 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:40.454364 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1409/2802] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:42:40.458215 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed5b2e-0268-477c-a5cc-bb6735f85597 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:40.469215 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed5b2e-0268-477c-a5cc-bb6735f85597 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:40.473906 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3b093-70eb-4bda-a940-2db5e2fbb252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:40.474571 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3b093-70eb-4bda-a940-2db5e2fbb252 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:40.475363 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3b093-70eb-4bda-a940-2db5e2fbb252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:40.475363 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3b093-70eb-4bda-a940-2db5e2fbb252 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:40.475862 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3b093-70eb-4bda-a940-2db5e2fbb252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:40.477503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3b093-70eb-4bda-a940-2db5e2fbb252 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:40.487807 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1394/2803] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:42:40.492006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b8280-51a4-4257-b2b7-4368e315f1d3 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:40.502053 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b8280-51a4-4257-b2b7-4368e315f1d3 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:40.506629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e648c8c-57fb-4ee2-acfb-96e6e5e65870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:40.507503 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e648c8c-57fb-4ee2-acfb-96e6e5e65870 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:40.507785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e648c8c-57fb-4ee2-acfb-96e6e5e65870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:40.508061 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e648c8c-57fb-4ee2-acfb-96e6e5e65870 admin admin] PATH_INFO: `/v3/projects/d4e0cbdb77d04cc1a9df24f69f476466/users/ad34817d70154cb8bf13c3e87bb48991/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:40.522850 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e648c8c-57fb-4ee2-acfb-96e6e5e65870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4e0cbdb77d04cc1a9df24f69f476466, user_id=ad34817d70154cb8bf13c3e87bb48991, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:40.525536 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e648c8c-57fb-4ee2-acfb-96e6e5e65870 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:40.539197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1410/2804] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:42:40 2022] PUT /identity/v3/projects/d4e0cbdb77d04cc1a9df24f69f476466/users/ad34817d70154cb8bf13c3e87bb48991/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:40.549476 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a64a49-7193-43f2-8741-6d74b5775bc1 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:40.549774 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a64a49-7193-43f2-8741-6d74b5775bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:40.550084 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a64a49-7193-43f2-8741-6d74b5775bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:40.577396 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d8a64a49-7193-43f2-8741-6d74b5775bc1 None None] MFA Rules not processed for user `ad34817d70154cb8bf13c3e87bb48991`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:40.578641 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d8a64a49-7193-43f2-8741-6d74b5775bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:40.592059 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:40.644626 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1395/2805] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:40.649126 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e648c8c-57fb-4ee2-acfb-96e6e5e65870 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:40.712741 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e648c8c-57fb-4ee2-acfb-96e6e5e65870 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:40.718427 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba39a70-f5a9-4855-86d9-d90c531f5dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:40.719098 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba39a70-f5a9-4855-86d9-d90c531f5dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:40.719380 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba39a70-f5a9-4855-86d9-d90c531f5dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:40.719661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba39a70-f5a9-4855-86d9-d90c531f5dd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:40.720359 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba39a70-f5a9-4855-86d9-d90c531f5dd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:40.722309 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba39a70-f5a9-4855-86d9-d90c531f5dd2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:40.747523 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1411/2806] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:42:40 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:42:40.751481 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a64a49-7193-43f2-8741-6d74b5775bc1 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:40.762621 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a64a49-7193-43f2-8741-6d74b5775bc1 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:40.818285 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0e4f5-8d79-498b-8cbb-29c3b285bc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:40.818961 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0e4f5-8d79-498b-8cbb-29c3b285bc56 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:40.819250 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0e4f5-8d79-498b-8cbb-29c3b285bc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:40.819527 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0e4f5-8d79-498b-8cbb-29c3b285bc56 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:40.820226 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0e4f5-8d79-498b-8cbb-29c3b285bc56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:40.822242 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0e4f5-8d79-498b-8cbb-29c3b285bc56 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:40.843305 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1396/2807] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:42:40 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:42:40.847180 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba39a70-f5a9-4855-86d9-d90c531f5dd2 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:40.857868 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba39a70-f5a9-4855-86d9-d90c531f5dd2 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:40.862555 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b9b4f-a493-49c5-923d-fade9361f632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:40.863214 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b9b4f-a493-49c5-923d-fade9361f632 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:40.863497 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b9b4f-a493-49c5-923d-fade9361f632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:40.863774 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b9b4f-a493-49c5-923d-fade9361f632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:40.864499 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b9b4f-a493-49c5-923d-fade9361f632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:40.866145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b9b4f-a493-49c5-923d-fade9361f632 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:40.877442 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1412/2808] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:42:40.881555 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0e4f5-8d79-498b-8cbb-29c3b285bc56 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:40.892312 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0e4f5-8d79-498b-8cbb-29c3b285bc56 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:40.896942 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae23789-067e-412a-898f-9626b1b2909d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:40.897799 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae23789-067e-412a-898f-9626b1b2909d admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:40.898090 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae23789-067e-412a-898f-9626b1b2909d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:40.898368 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae23789-067e-412a-898f-9626b1b2909d admin admin] PATH_INFO: `/v3/projects/2b3a4d4abae04e138a7cf152c0ef0294/users/fb14df67a20b4a62b69d64e7df5133d4/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:40.913967 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae23789-067e-412a-898f-9626b1b2909d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3a4d4abae04e138a7cf152c0ef0294, user_id=fb14df67a20b4a62b69d64e7df5133d4, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:40.916611 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae23789-067e-412a-898f-9626b1b2909d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:40.930367 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1397/2809] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:42:40 2022] PUT /identity/v3/projects/2b3a4d4abae04e138a7cf152c0ef0294/users/fb14df67a20b4a62b69d64e7df5133d4/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:40.934834 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b9b4f-a493-49c5-923d-fade9361f632 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:40.944639 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b9b4f-a493-49c5-923d-fade9361f632 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:41.003202 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10792549-e71c-418a-be5f-d2a18657e857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:41.003909 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10792549-e71c-418a-be5f-d2a18657e857 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:41.004194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10792549-e71c-418a-be5f-d2a18657e857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:41.004471 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10792549-e71c-418a-be5f-d2a18657e857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:41.005206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10792549-e71c-418a-be5f-d2a18657e857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:41.006830 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10792549-e71c-418a-be5f-d2a18657e857 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:41.017572 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1413/2810] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:42:41.021840 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae23789-067e-412a-898f-9626b1b2909d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:41.032405 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae23789-067e-412a-898f-9626b1b2909d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:41.037407 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee87af-344a-4a44-b852-5eaf5e3c1fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:41.038300 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee87af-344a-4a44-b852-5eaf5e3c1fe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:41.038581 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee87af-344a-4a44-b852-5eaf5e3c1fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:41.038862 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee87af-344a-4a44-b852-5eaf5e3c1fe6 admin admin] PATH_INFO: `/v3/projects/2b3a4d4abae04e138a7cf152c0ef0294/users/fb14df67a20b4a62b69d64e7df5133d4/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:41.040827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee87af-344a-4a44-b852-5eaf5e3c1fe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3a4d4abae04e138a7cf152c0ef0294, user_id=fb14df67a20b4a62b69d64e7df5133d4, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:41.043783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee87af-344a-4a44-b852-5eaf5e3c1fe6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:41.054369 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1398/2811] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:42:41 2022] PUT /identity/v3/projects/2b3a4d4abae04e138a7cf152c0ef0294/users/fb14df67a20b4a62b69d64e7df5133d4/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:41.066209 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded33c7-4045-44f8-a7df-7fe7ab66f183 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:41.066494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded33c7-4045-44f8-a7df-7fe7ab66f183 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:41.066769 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded33c7-4045-44f8-a7df-7fe7ab66f183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:41.093511 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-3ded33c7-4045-44f8-a7df-7fe7ab66f183 None None] MFA Rules not processed for user `fb14df67a20b4a62b69d64e7df5133d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:41.094719 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3ded33c7-4045-44f8-a7df-7fe7ab66f183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:41.129175 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:41.165083 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1414/2812] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:41.176308 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee87af-344a-4a44-b852-5eaf5e3c1fe6 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:41.178283 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-41ee87af-344a-4a44-b852-5eaf5e3c1fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:41.236025 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee87af-344a-4a44-b852-5eaf5e3c1fe6 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:41.239656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfcd80c-de4e-4ab2-ac4d-ce9dc353d3d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:41.240301 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfcd80c-de4e-4ab2-ac4d-ce9dc353d3d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:41.240589 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfcd80c-de4e-4ab2-ac4d-ce9dc353d3d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:41.240869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfcd80c-de4e-4ab2-ac4d-ce9dc353d3d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:41.262754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:41.303149 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfcd80c-de4e-4ab2-ac4d-ce9dc353d3d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:41.304783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfcd80c-de4e-4ab2-ac4d-ce9dc353d3d8 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:41.315290 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1399/2813] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:42:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:42:45.471548 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded33c7-4045-44f8-a7df-7fe7ab66f183 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:45.473037 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-3ded33c7-4045-44f8-a7df-7fe7ab66f183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:45.486237 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded33c7-4045-44f8-a7df-7fe7ab66f183 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:45.490304 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b4b7b-cb7a-4479-954b-55d7a235b486 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:45.490971 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b4b7b-cb7a-4479-954b-55d7a235b486 service cinder] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:45.491255 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b4b7b-cb7a-4479-954b-55d7a235b486 service cinder] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:45.491527 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b4b7b-cb7a-4479-954b-55d7a235b486 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:45.501521 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b4b7b-cb7a-4479-954b-55d7a235b486 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:45.502966 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b4b7b-cb7a-4479-954b-55d7a235b486 service cinder] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:45.512554 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1415/2814] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:42:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:42:50.389700 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfcd80c-de4e-4ab2-ac4d-ce9dc353d3d8 service cinder] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:50.450994 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfcd80c-de4e-4ab2-ac4d-ce9dc353d3d8 service cinder] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:50.455459 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59c49d-80c7-429f-88ab-7d86c9cb9494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:50.456319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59c49d-80c7-429f-88ab-7d86c9cb9494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:50.456604 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59c49d-80c7-429f-88ab-7d86c9cb9494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:50.456881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59c49d-80c7-429f-88ab-7d86c9cb9494 admin admin] PATH_INFO: `/v3/users/ad34817d70154cb8bf13c3e87bb48991` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:50.458079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59c49d-80c7-429f-88ab-7d86c9cb9494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad34817d70154cb8bf13c3e87bb48991)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:50.459987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59c49d-80c7-429f-88ab-7d86c9cb9494 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:50.504769 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7d59c49d-80c7-429f-88ab-7d86c9cb9494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad34817d70154cb8bf13c3e87bb48991'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:50.517210 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7d59c49d-80c7-429f-88ab-7d86c9cb9494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad34817d70154cb8bf13c3e87bb48991'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:50.524270 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7d59c49d-80c7-429f-88ab-7d86c9cb9494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad34817d70154cb8bf13c3e87bb48991'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:50.524877 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7d59c49d-80c7-429f-88ab-7d86c9cb9494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad34817d70154cb8bf13c3e87bb48991'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:50.525492 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-7d59c49d-80c7-429f-88ab-7d86c9cb9494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad34817d70154cb8bf13c3e87bb48991'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:50.541573 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1400/2815] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:42:50 2022] DELETE /identity/v3/users/ad34817d70154cb8bf13c3e87bb48991 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:50.546664 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d1a9a-49ee-4006-85dc-f1dddb0852b1 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:50.547194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d1a9a-49ee-4006-85dc-f1dddb0852b1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:50.547194 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d1a9a-49ee-4006-85dc-f1dddb0852b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:50.565893 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-993d1a9a-49ee-4006-85dc-f1dddb0852b1 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:50.567173 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-993d1a9a-49ee-4006-85dc-f1dddb0852b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:50.625819 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1416/2816] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:50.635205 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59c49d-80c7-429f-88ab-7d86c9cb9494 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:50.637515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-7d59c49d-80c7-429f-88ab-7d86c9cb9494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:50.682828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:50.698342 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59c49d-80c7-429f-88ab-7d86c9cb9494 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:50.702840 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5e05d-5aa1-4160-9683-7d67453852df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:50.703503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5e05d-5aa1-4160-9683-7d67453852df service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:50.703790 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5e05d-5aa1-4160-9683-7d67453852df service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:50.704074 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5e05d-5aa1-4160-9683-7d67453852df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:50.713690 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5e05d-5aa1-4160-9683-7d67453852df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:50.715846 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5e05d-5aa1-4160-9683-7d67453852df service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:50.725740 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1401/2817] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:42:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:42:50.994229 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d1a9a-49ee-4006-85dc-f1dddb0852b1 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:50.996823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-993d1a9a-49ee-4006-85dc-f1dddb0852b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:51.018914 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d1a9a-49ee-4006-85dc-f1dddb0852b1 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:51.026201 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:51.027839 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:51.028192 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:51.028526 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2 admin admin] PATH_INFO: `/v3/projects/d4e0cbdb77d04cc1a9df24f69f476466` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:51.029823 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4e0cbdb77d04cc1a9df24f69f476466)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:51.032127 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:51.078834 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4e0cbdb77d04cc1a9df24f69f476466', 'request_id': 'req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:51.084469 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4e0cbdb77d04cc1a9df24f69f476466', 'request_id': 'req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:51.092309 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4e0cbdb77d04cc1a9df24f69f476466', 'request_id': 'req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:51.103977 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2 admin admin] The token cache is being invalidate because project d4e0cbdb77d04cc1a9df24f69f476466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:42:51.104265 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:51.105019 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:51.106940 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1417/2818] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:42:50 2022] DELETE /identity/v3/projects/d4e0cbdb77d04cc1a9df24f69f476466 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:51.110816 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5e05d-5aa1-4160-9683-7d67453852df service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:51.120685 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5e05d-5aa1-4160-9683-7d67453852df service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:51.176898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0950e5-3dbc-426d-a1bc-cd1250388c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:51.177711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0950e5-3dbc-426d-a1bc-cd1250388c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:51.177995 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0950e5-3dbc-426d-a1bc-cd1250388c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:51.178275 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0950e5-3dbc-426d-a1bc-cd1250388c88 admin admin] PATH_INFO: `/v3/users/fb14df67a20b4a62b69d64e7df5133d4` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:51.179440 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0950e5-3dbc-426d-a1bc-cd1250388c88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb14df67a20b4a62b69d64e7df5133d4)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:51.181374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0950e5-3dbc-426d-a1bc-cd1250388c88 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:51.228471 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6f0950e5-3dbc-426d-a1bc-cd1250388c88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb14df67a20b4a62b69d64e7df5133d4'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:51.240434 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6f0950e5-3dbc-426d-a1bc-cd1250388c88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb14df67a20b4a62b69d64e7df5133d4'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:51.247875 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6f0950e5-3dbc-426d-a1bc-cd1250388c88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb14df67a20b4a62b69d64e7df5133d4'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:51.248501 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6f0950e5-3dbc-426d-a1bc-cd1250388c88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb14df67a20b4a62b69d64e7df5133d4'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:51.249102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-6f0950e5-3dbc-426d-a1bc-cd1250388c88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb14df67a20b4a62b69d64e7df5133d4'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:51.262235 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1402/2819] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:42:51 2022] DELETE /identity/v3/users/fb14df67a20b4a62b69d64e7df5133d4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:51.529236 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:51.530711 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:51.547595 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9780c9-7d62-471e-888a-1dbc12e1d3e2 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:51.590082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:51.609363 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669201a-1414-45d7-b3b4-0def83664a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:51.610234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669201a-1414-45d7-b3b4-0def83664a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:51.610520 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669201a-1414-45d7-b3b4-0def83664a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:51.610795 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669201a-1414-45d7-b3b4-0def83664a75 admin admin] PATH_INFO: `/v3/projects/2b3a4d4abae04e138a7cf152c0ef0294` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:51.611976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669201a-1414-45d7-b3b4-0def83664a75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b3a4d4abae04e138a7cf152c0ef0294)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:51.613923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669201a-1414-45d7-b3b4-0def83664a75 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:51.661565 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4669201a-1414-45d7-b3b4-0def83664a75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b3a4d4abae04e138a7cf152c0ef0294', 'request_id': 'req-4669201a-1414-45d7-b3b4-0def83664a75'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:51.667001 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4669201a-1414-45d7-b3b4-0def83664a75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b3a4d4abae04e138a7cf152c0ef0294', 'request_id': 'req-4669201a-1414-45d7-b3b4-0def83664a75'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:51.674336 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4669201a-1414-45d7-b3b4-0def83664a75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b3a4d4abae04e138a7cf152c0ef0294', 'request_id': 'req-4669201a-1414-45d7-b3b4-0def83664a75'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:51.685633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4669201a-1414-45d7-b3b4-0def83664a75 admin admin] The token cache is being invalidate because project 2b3a4d4abae04e138a7cf152c0ef0294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80280) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:42:51.685921 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4669201a-1414-45d7-b3b4-0def83664a75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:51.686547 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-4669201a-1414-45d7-b3b4-0def83664a75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:51.688135 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1418/2820] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:42:51 2022] DELETE /identity/v3/projects/2b3a4d4abae04e138a7cf152c0ef0294 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:51.694142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acc8be-387d-42b8-8831-0e41a078517c None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:51.694655 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acc8be-387d-42b8-8831-0e41a078517c None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:51.694745 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acc8be-387d-42b8-8831-0e41a078517c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:51.713969 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-f7acc8be-387d-42b8-8831-0e41a078517c None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:51.715899 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-f7acc8be-387d-42b8-8831-0e41a078517c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:51.778108 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1403/2821] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 06:42:51.786815 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946867c1-2f30-4162-8850-d5e6618736b3 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:51.787112 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946867c1-2f30-4162-8850-d5e6618736b3 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:51.787396 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946867c1-2f30-4162-8850-d5e6618736b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:51.806312 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-946867c1-2f30-4162-8850-d5e6618736b3 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:51.807531 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-946867c1-2f30-4162-8850-d5e6618736b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:51.817735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1419/2822] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:51.822030 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7acc8be-387d-42b8-8831-0e41a078517c None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:51.828733 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:51.832781 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7acc8be-387d-42b8-8831-0e41a078517c None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:51.838617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88a583-b2fe-4784-95a7-a04597a0150d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:51.840372 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88a583-b2fe-4784-95a7-a04597a0150d admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:51.840883 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88a583-b2fe-4784-95a7-a04597a0150d admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:51.841553 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88a583-b2fe-4784-95a7-a04597a0150d admin admin] PATH_INFO: `/v3/domains` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:51.842474 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88a583-b2fe-4784-95a7-a04597a0150d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:42:51.842790 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88a583-b2fe-4784-95a7-a04597a0150d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:51.844447 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88a583-b2fe-4784-95a7-a04597a0150d admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:51.862932 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1404/2823] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:42:51.867633 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946867c1-2f30-4162-8850-d5e6618736b3 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:51.876992 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946867c1-2f30-4162-8850-d5e6618736b3 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:51.881721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddea6c8-a398-4520-9d25-b2163dcea5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:51.882376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddea6c8-a398-4520-9d25-b2163dcea5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:51.882660 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddea6c8-a398-4520-9d25-b2163dcea5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:51.882939 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddea6c8-a398-4520-9d25-b2163dcea5a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:51.883641 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddea6c8-a398-4520-9d25-b2163dcea5a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:51.885604 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddea6c8-a398-4520-9d25-b2163dcea5a4 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:51.912819 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1420/2824] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:42:51 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:42:51.917968 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88a583-b2fe-4784-95a7-a04597a0150d admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:51.928551 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88a583-b2fe-4784-95a7-a04597a0150d admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:51.985213 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55b3c4-f5d6-4097-a7d6-ea419dc3d641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:51.985831 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55b3c4-f5d6-4097-a7d6-ea419dc3d641 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:51.986116 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55b3c4-f5d6-4097-a7d6-ea419dc3d641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:51.986393 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55b3c4-f5d6-4097-a7d6-ea419dc3d641 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:51.987087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55b3c4-f5d6-4097-a7d6-ea419dc3d641 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:51.989010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55b3c4-f5d6-4097-a7d6-ea419dc3d641 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:52.012172 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1405/2825] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:42:51 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:42:52.013397 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddea6c8-a398-4520-9d25-b2163dcea5a4 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:52.023491 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddea6c8-a398-4520-9d25-b2163dcea5a4 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:52.028096 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79576fb1-490b-4cd6-8634-be792d730e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:52.028750 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79576fb1-490b-4cd6-8634-be792d730e28 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:52.029033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79576fb1-490b-4cd6-8634-be792d730e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:52.029340 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79576fb1-490b-4cd6-8634-be792d730e28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:52.030070 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79576fb1-490b-4cd6-8634-be792d730e28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:52.031661 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79576fb1-490b-4cd6-8634-be792d730e28 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:52.042216 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1421/2826] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:42:52.046437 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55b3c4-f5d6-4097-a7d6-ea419dc3d641 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:52.056533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55b3c4-f5d6-4097-a7d6-ea419dc3d641 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:52.061709 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e51c5-ae98-4e1e-99a4-4d821c7d86a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:52.062371 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e51c5-ae98-4e1e-99a4-4d821c7d86a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:52.062673 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e51c5-ae98-4e1e-99a4-4d821c7d86a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:52.062939 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e51c5-ae98-4e1e-99a4-4d821c7d86a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:52.063881 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e51c5-ae98-4e1e-99a4-4d821c7d86a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:52.065317 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e51c5-ae98-4e1e-99a4-4d821c7d86a9 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:52.075455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1406/2827] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:42:52.079598 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79576fb1-490b-4cd6-8634-be792d730e28 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:52.092574 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79576fb1-490b-4cd6-8634-be792d730e28 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:52.101090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddd707-d078-4c5d-b8ce-a206aa60fd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:52.102378 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddd707-d078-4c5d-b8ce-a206aa60fd13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:52.102671 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddd707-d078-4c5d-b8ce-a206aa60fd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:52.102955 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddd707-d078-4c5d-b8ce-a206aa60fd13 admin admin] PATH_INFO: `/v3/projects/4fb683232a344808aed0da55c5163352/users/1fe18c906ecf4092a3ff1a1f047e4f7b/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:52.118714 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddd707-d078-4c5d-b8ce-a206aa60fd13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fb683232a344808aed0da55c5163352, user_id=1fe18c906ecf4092a3ff1a1f047e4f7b, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:52.121370 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddd707-d078-4c5d-b8ce-a206aa60fd13 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:52.134042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1422/2828] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:42:52 2022] PUT /identity/v3/projects/4fb683232a344808aed0da55c5163352/users/1fe18c906ecf4092a3ff1a1f047e4f7b/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:52.145493 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb442f-db1f-462a-b5a9-fe453e8ba7bb None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:52.145788 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb442f-db1f-462a-b5a9-fe453e8ba7bb None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:52.146076 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb442f-db1f-462a-b5a9-fe453e8ba7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:52.174283 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-8deb442f-db1f-462a-b5a9-fe453e8ba7bb None None] MFA Rules not processed for user `1fe18c906ecf4092a3ff1a1f047e4f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:52.175489 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-8deb442f-db1f-462a-b5a9-fe453e8ba7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:52.242374 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1407/2829] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:52.247062 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddd707-d078-4c5d-b8ce-a206aa60fd13 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:52.308416 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddd707-d078-4c5d-b8ce-a206aa60fd13 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:52.313193 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57ca47-5ef7-4448-b53c-1ebf9f46d56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:52.313884 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57ca47-5ef7-4448-b53c-1ebf9f46d56e admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:52.314171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57ca47-5ef7-4448-b53c-1ebf9f46d56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:52.314450 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57ca47-5ef7-4448-b53c-1ebf9f46d56e admin admin] PATH_INFO: `/v3/projects` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:52.315145 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57ca47-5ef7-4448-b53c-1ebf9f46d56e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:52.317049 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57ca47-5ef7-4448-b53c-1ebf9f46d56e admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:52.330472 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:52.348024 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1423/2830] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:42:52 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:42:52.349734 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb442f-db1f-462a-b5a9-fe453e8ba7bb None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:52.360975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb442f-db1f-462a-b5a9-fe453e8ba7bb None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:52.418216 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6f0cf-940a-41a4-ab23-fb326413a037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:52.418880 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6f0cf-940a-41a4-ab23-fb326413a037 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:52.419169 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6f0cf-940a-41a4-ab23-fb326413a037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:52.419437 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6f0cf-940a-41a4-ab23-fb326413a037 admin admin] PATH_INFO: `/v3/users` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:52.420120 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6f0cf-940a-41a4-ab23-fb326413a037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:52.422090 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6f0cf-940a-41a4-ab23-fb326413a037 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:52.428949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:52.443074 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1408/2831] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:42:52 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:42:52.447086 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57ca47-5ef7-4448-b53c-1ebf9f46d56e admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:52.458115 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57ca47-5ef7-4448-b53c-1ebf9f46d56e admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:52.462735 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f79f48-02d4-4e76-a9d3-84134ff185b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:52.463433 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f79f48-02d4-4e76-a9d3-84134ff185b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:52.463721 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f79f48-02d4-4e76-a9d3-84134ff185b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:52.464002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f79f48-02d4-4e76-a9d3-84134ff185b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:52.465467 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f79f48-02d4-4e76-a9d3-84134ff185b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:52.467089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f79f48-02d4-4e76-a9d3-84134ff185b7 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:52.478457 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1424/2832] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:42:52.482656 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6f0cf-940a-41a4-ab23-fb326413a037 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:52.493065 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6f0cf-940a-41a4-ab23-fb326413a037 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:52.497727 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a8085-8a9b-4f5a-a857-a9b88fcda10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:52.498610 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a8085-8a9b-4f5a-a857-a9b88fcda10f admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:52.498897 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a8085-8a9b-4f5a-a857-a9b88fcda10f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:52.499176 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a8085-8a9b-4f5a-a857-a9b88fcda10f admin admin] PATH_INFO: `/v3/projects/b03ea02417904a2fb9fac41d85b51faa/users/513b76264de341888817b000eede60d1/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:52.514430 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a8085-8a9b-4f5a-a857-a9b88fcda10f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b03ea02417904a2fb9fac41d85b51faa, user_id=513b76264de341888817b000eede60d1, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:52.517091 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a8085-8a9b-4f5a-a857-a9b88fcda10f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:52.530164 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1409/2833] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:42:52 2022] PUT /identity/v3/projects/b03ea02417904a2fb9fac41d85b51faa/users/513b76264de341888817b000eede60d1/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:52.534454 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f79f48-02d4-4e76-a9d3-84134ff185b7 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:52.545155 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f79f48-02d4-4e76-a9d3-84134ff185b7 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:52.600899 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd4f96-6e36-4e74-9d0a-7f5d20e54315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:52.601722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd4f96-6e36-4e74-9d0a-7f5d20e54315 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:52.601997 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd4f96-6e36-4e74-9d0a-7f5d20e54315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:52.602276 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd4f96-6e36-4e74-9d0a-7f5d20e54315 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:52.603005 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd4f96-6e36-4e74-9d0a-7f5d20e54315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:52.604602 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd4f96-6e36-4e74-9d0a-7f5d20e54315 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:52.614671 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1425/2834] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:42:52.618789 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a8085-8a9b-4f5a-a857-a9b88fcda10f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:52.629306 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a8085-8a9b-4f5a-a857-a9b88fcda10f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:52.634251 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b6faf-2eea-43c6-9175-9e5d4d8fa19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:52.635132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b6faf-2eea-43c6-9175-9e5d4d8fa19c admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:52.635421 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b6faf-2eea-43c6-9175-9e5d4d8fa19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:52.635705 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b6faf-2eea-43c6-9175-9e5d4d8fa19c admin admin] PATH_INFO: `/v3/projects/b03ea02417904a2fb9fac41d85b51faa/users/513b76264de341888817b000eede60d1/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:52.638351 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b6faf-2eea-43c6-9175-9e5d4d8fa19c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b03ea02417904a2fb9fac41d85b51faa, user_id=513b76264de341888817b000eede60d1, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:52.641002 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b6faf-2eea-43c6-9175-9e5d4d8fa19c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:52.651401 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1410/2835] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:42:52 2022] PUT /identity/v3/projects/b03ea02417904a2fb9fac41d85b51faa/users/513b76264de341888817b000eede60d1/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:52.713613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88ceac-dadd-41e4-b611-9ee798f4ebff None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:52.713902 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88ceac-dadd-41e4-b611-9ee798f4ebff None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:52.714183 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88ceac-dadd-41e4-b611-9ee798f4ebff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:52.740447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ab88ceac-dadd-41e4-b611-9ee798f4ebff None None] MFA Rules not processed for user `513b76264de341888817b000eede60d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:52.741676 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ab88ceac-dadd-41e4-b611-9ee798f4ebff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:52.810042 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1426/2836] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:52.821221 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b6faf-2eea-43c6-9175-9e5d4d8fa19c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:52.823077 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-b01b6faf-2eea-43c6-9175-9e5d4d8fa19c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:52.887119 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b6faf-2eea-43c6-9175-9e5d4d8fa19c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:52.891018 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55380b2d-1edf-4a9c-bf28-57a8f06a8acf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:52.891777 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55380b2d-1edf-4a9c-bf28-57a8f06a8acf service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:52.892151 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55380b2d-1edf-4a9c-bf28-57a8f06a8acf service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:52.892522 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55380b2d-1edf-4a9c-bf28-57a8f06a8acf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:52.955654 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55380b2d-1edf-4a9c-bf28-57a8f06a8acf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:52.957106 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55380b2d-1edf-4a9c-bf28-57a8f06a8acf service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:52.966037 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1411/2837] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:42:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:42:55.485036 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88ceac-dadd-41e4-b611-9ee798f4ebff None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:55.549093 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88ceac-dadd-41e4-b611-9ee798f4ebff None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:55.553613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35d99d-3c3c-4863-b5cb-8f867e7aff30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:55.554481 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35d99d-3c3c-4863-b5cb-8f867e7aff30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:55.554759 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35d99d-3c3c-4863-b5cb-8f867e7aff30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:55.555031 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35d99d-3c3c-4863-b5cb-8f867e7aff30 admin admin] PATH_INFO: `/v3/users/1fe18c906ecf4092a3ff1a1f047e4f7b` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:55.556206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac35d99d-3c3c-4863-b5cb-8f867e7aff30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fe18c906ecf4092a3ff1a1f047e4f7b)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:55.558861 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac35d99d-3c3c-4863-b5cb-8f867e7aff30 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:55.563819 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:55.608714 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ac35d99d-3c3c-4863-b5cb-8f867e7aff30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fe18c906ecf4092a3ff1a1f047e4f7b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:55.609309 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ac35d99d-3c3c-4863-b5cb-8f867e7aff30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fe18c906ecf4092a3ff1a1f047e4f7b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:55.621813 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ac35d99d-3c3c-4863-b5cb-8f867e7aff30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fe18c906ecf4092a3ff1a1f047e4f7b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:55.622461 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ac35d99d-3c3c-4863-b5cb-8f867e7aff30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fe18c906ecf4092a3ff1a1f047e4f7b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:55.629694 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-ac35d99d-3c3c-4863-b5cb-8f867e7aff30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fe18c906ecf4092a3ff1a1f047e4f7b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:55.641945 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1427/2838] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:42:55 2022] DELETE /identity/v3/users/1fe18c906ecf4092a3ff1a1f047e4f7b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:55.646503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880737e-4547-49d2-a485-2b39790a4d93 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:55.646996 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880737e-4547-49d2-a485-2b39790a4d93 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:55.647099 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880737e-4547-49d2-a485-2b39790a4d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:55.664595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-d880737e-4547-49d2-a485-2b39790a4d93 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:55.665839 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d880737e-4547-49d2-a485-2b39790a4d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:55.725658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1412/2839] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:55.735753 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35d99d-3c3c-4863-b5cb-8f867e7aff30 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:55.737151 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ac35d99d-3c3c-4863-b5cb-8f867e7aff30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:55.795544 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35d99d-3c3c-4863-b5cb-8f867e7aff30 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:55.799251 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cc255-9d1a-489c-83e6-9e311c411383 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:55.799900 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cc255-9d1a-489c-83e6-9e311c411383 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:55.800182 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cc255-9d1a-489c-83e6-9e311c411383 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:55.800458 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cc255-9d1a-489c-83e6-9e311c411383 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:55.809530 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598cc255-9d1a-489c-83e6-9e311c411383 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:55.810962 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598cc255-9d1a-489c-83e6-9e311c411383 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:55.820376 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1428/2840] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:42:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:42:56.095406 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880737e-4547-49d2-a485-2b39790a4d93 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:56.097989 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d880737e-4547-49d2-a485-2b39790a4d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:56.111726 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:56.143144 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880737e-4547-49d2-a485-2b39790a4d93 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:56.148754 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af519ebf-a570-4f19-963d-aebd34626811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:56.149650 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af519ebf-a570-4f19-963d-aebd34626811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:56.149935 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af519ebf-a570-4f19-963d-aebd34626811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:56.150211 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af519ebf-a570-4f19-963d-aebd34626811 admin admin] PATH_INFO: `/v3/projects/4fb683232a344808aed0da55c5163352` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:56.151378 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af519ebf-a570-4f19-963d-aebd34626811 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fb683232a344808aed0da55c5163352)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:56.154003 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af519ebf-a570-4f19-963d-aebd34626811 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:56.213230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-af519ebf-a570-4f19-963d-aebd34626811 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fb683232a344808aed0da55c5163352', 'request_id': 'req-af519ebf-a570-4f19-963d-aebd34626811'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:56.222573 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-af519ebf-a570-4f19-963d-aebd34626811 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fb683232a344808aed0da55c5163352', 'request_id': 'req-af519ebf-a570-4f19-963d-aebd34626811'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:56.228283 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-af519ebf-a570-4f19-963d-aebd34626811 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fb683232a344808aed0da55c5163352', 'request_id': 'req-af519ebf-a570-4f19-963d-aebd34626811'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:56.240026 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-af519ebf-a570-4f19-963d-aebd34626811 admin admin] The token cache is being invalidate because project 4fb683232a344808aed0da55c5163352 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:42:56.240311 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-af519ebf-a570-4f19-963d-aebd34626811 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:56.240951 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-af519ebf-a570-4f19-963d-aebd34626811 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:56.242519 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1413/2841] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:42:56 2022] DELETE /identity/v3/projects/4fb683232a344808aed0da55c5163352 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:56.246748 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cc255-9d1a-489c-83e6-9e311c411383 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:56.266885 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cc255-9d1a-489c-83e6-9e311c411383 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:56.323135 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8b099-5b32-4c5e-9f48-9a1fc12d2611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:56.324634 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8b099-5b32-4c5e-9f48-9a1fc12d2611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:56.324922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8b099-5b32-4c5e-9f48-9a1fc12d2611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:56.325197 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8b099-5b32-4c5e-9f48-9a1fc12d2611 admin admin] PATH_INFO: `/v3/users/513b76264de341888817b000eede60d1` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:56.326431 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8b099-5b32-4c5e-9f48-9a1fc12d2611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=513b76264de341888817b000eede60d1)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:56.328351 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8b099-5b32-4c5e-9f48-9a1fc12d2611 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:56.351022 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:56.377678 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-72d8b099-5b32-4c5e-9f48-9a1fc12d2611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '513b76264de341888817b000eede60d1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:56.378330 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-72d8b099-5b32-4c5e-9f48-9a1fc12d2611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '513b76264de341888817b000eede60d1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:56.390579 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-72d8b099-5b32-4c5e-9f48-9a1fc12d2611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '513b76264de341888817b000eede60d1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:56.391243 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-72d8b099-5b32-4c5e-9f48-9a1fc12d2611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '513b76264de341888817b000eede60d1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:56.398147 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-72d8b099-5b32-4c5e-9f48-9a1fc12d2611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '513b76264de341888817b000eede60d1'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:56.411754 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1429/2842] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:42:56 2022] DELETE /identity/v3/users/513b76264de341888817b000eede60d1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:56.681480 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af519ebf-a570-4f19-963d-aebd34626811 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:56.682892 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-af519ebf-a570-4f19-963d-aebd34626811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:56.696318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af519ebf-a570-4f19-963d-aebd34626811 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:56.754755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cae4e-a1a5-4080-ac20-86b2ebf0d196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:56.755620 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cae4e-a1a5-4080-ac20-86b2ebf0d196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:56.755904 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cae4e-a1a5-4080-ac20-86b2ebf0d196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:56.756184 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cae4e-a1a5-4080-ac20-86b2ebf0d196 admin admin] PATH_INFO: `/v3/projects/b03ea02417904a2fb9fac41d85b51faa` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:56.757380 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600cae4e-a1a5-4080-ac20-86b2ebf0d196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b03ea02417904a2fb9fac41d85b51faa)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:56.759278 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600cae4e-a1a5-4080-ac20-86b2ebf0d196 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:56.806030 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-600cae4e-a1a5-4080-ac20-86b2ebf0d196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b03ea02417904a2fb9fac41d85b51faa', 'request_id': 'req-600cae4e-a1a5-4080-ac20-86b2ebf0d196'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:56.813270 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-600cae4e-a1a5-4080-ac20-86b2ebf0d196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b03ea02417904a2fb9fac41d85b51faa', 'request_id': 'req-600cae4e-a1a5-4080-ac20-86b2ebf0d196'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:56.818762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-600cae4e-a1a5-4080-ac20-86b2ebf0d196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b03ea02417904a2fb9fac41d85b51faa', 'request_id': 'req-600cae4e-a1a5-4080-ac20-86b2ebf0d196'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:56.832130 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-600cae4e-a1a5-4080-ac20-86b2ebf0d196 admin admin] The token cache is being invalidate because project b03ea02417904a2fb9fac41d85b51faa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:42:56.832441 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-600cae4e-a1a5-4080-ac20-86b2ebf0d196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:56.833149 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-600cae4e-a1a5-4080-ac20-86b2ebf0d196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:56.834808 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1414/2843] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:42:56 2022] DELETE /identity/v3/projects/b03ea02417904a2fb9fac41d85b51faa => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:56.840436 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4085f3-ba0c-4c38-ab34-6d8cea812acc None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:56.840923 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4085f3-ba0c-4c38-ab34-6d8cea812acc None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:56.841033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4085f3-ba0c-4c38-ab34-6d8cea812acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:56.860089 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-fd4085f3-ba0c-4c38-ab34-6d8cea812acc None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:56.861312 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fd4085f3-ba0c-4c38-ab34-6d8cea812acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:56.921785 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1430/2844] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:56.929515 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533976c3-cbe8-45ce-8db6-ba1d27304367 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:56.929967 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533976c3-cbe8-45ce-8db6-ba1d27304367 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:56.930353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533976c3-cbe8-45ce-8db6-ba1d27304367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:56.937575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:56.961699 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-533976c3-cbe8-45ce-8db6-ba1d27304367 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:56.962945 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-533976c3-cbe8-45ce-8db6-ba1d27304367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:56.971635 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1415/2845] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:56.975824 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4085f3-ba0c-4c38-ab34-6d8cea812acc None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:56.985655 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4085f3-ba0c-4c38-ab34-6d8cea812acc None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:56.990198 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b96b9-8107-4934-b435-00a4dca6cb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:56.990858 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b96b9-8107-4934-b435-00a4dca6cb65 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:56.991147 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b96b9-8107-4934-b435-00a4dca6cb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:56.991429 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b96b9-8107-4934-b435-00a4dca6cb65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:56.992242 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b96b9-8107-4934-b435-00a4dca6cb65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:42:56.992557 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b96b9-8107-4934-b435-00a4dca6cb65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:56.994192 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b96b9-8107-4934-b435-00a4dca6cb65 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:57.006143 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1431/2846] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:42:57.010540 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533976c3-cbe8-45ce-8db6-ba1d27304367 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:57.020453 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533976c3-cbe8-45ce-8db6-ba1d27304367 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:57.025039 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64197aa0-893f-46f7-9fca-70af46144725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:57.025719 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64197aa0-893f-46f7-9fca-70af46144725 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:57.026010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64197aa0-893f-46f7-9fca-70af46144725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:57.026284 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64197aa0-893f-46f7-9fca-70af46144725 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:57.027521 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64197aa0-893f-46f7-9fca-70af46144725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:57.029423 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64197aa0-893f-46f7-9fca-70af46144725 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:57.054380 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1416/2847] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:42:57 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:42:57.058649 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b96b9-8107-4934-b435-00a4dca6cb65 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:57.068828 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b96b9-8107-4934-b435-00a4dca6cb65 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:57.125997 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dfbe1f-e901-42ca-869e-aaf7f9f15648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:57.126703 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfbe1f-e901-42ca-869e-aaf7f9f15648 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:57.126988 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfbe1f-e901-42ca-869e-aaf7f9f15648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:57.127268 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfbe1f-e901-42ca-869e-aaf7f9f15648 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:57.127956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dfbe1f-e901-42ca-869e-aaf7f9f15648 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:57.129922 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dfbe1f-e901-42ca-869e-aaf7f9f15648 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:57.149931 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1432/2848] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:42:57 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:42:57.154919 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64197aa0-893f-46f7-9fca-70af46144725 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:57.165618 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64197aa0-893f-46f7-9fca-70af46144725 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:57.170309 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbb12f-1466-453e-b1d6-53c2b33d2ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:57.170965 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbb12f-1466-453e-b1d6-53c2b33d2ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:57.171249 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbb12f-1466-453e-b1d6-53c2b33d2ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:57.171524 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbb12f-1466-453e-b1d6-53c2b33d2ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:57.172238 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbb12f-1466-453e-b1d6-53c2b33d2ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:57.173853 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbb12f-1466-453e-b1d6-53c2b33d2ed1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:57.184308 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1417/2849] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:42:57.188518 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dfbe1f-e901-42ca-869e-aaf7f9f15648 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:57.199968 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dfbe1f-e901-42ca-869e-aaf7f9f15648 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:57.204701 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0d4f6-40c4-46df-b48a-c7aef35dcb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:57.205402 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0d4f6-40c4-46df-b48a-c7aef35dcb8d admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:57.205689 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0d4f6-40c4-46df-b48a-c7aef35dcb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:57.205971 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0d4f6-40c4-46df-b48a-c7aef35dcb8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:57.206715 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0d4f6-40c4-46df-b48a-c7aef35dcb8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:57.208312 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0d4f6-40c4-46df-b48a-c7aef35dcb8d admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:57.222129 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1433/2850] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:42:57.226348 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbb12f-1466-453e-b1d6-53c2b33d2ed1 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:57.236431 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbb12f-1466-453e-b1d6-53c2b33d2ed1 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:57.240999 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9183da-68a1-4be6-94fd-feadeb09e6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:57.241906 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9183da-68a1-4be6-94fd-feadeb09e6fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:57.242192 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9183da-68a1-4be6-94fd-feadeb09e6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:57.242469 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9183da-68a1-4be6-94fd-feadeb09e6fa admin admin] PATH_INFO: `/v3/projects/dd4f2a35474c48dcbdf6aba115a86f62/users/96c63fe260a34048961a90001052e647/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:57.257046 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9183da-68a1-4be6-94fd-feadeb09e6fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd4f2a35474c48dcbdf6aba115a86f62, user_id=96c63fe260a34048961a90001052e647, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:57.259706 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9183da-68a1-4be6-94fd-feadeb09e6fa admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:57.273811 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1418/2851] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:42:57 2022] PUT /identity/v3/projects/dd4f2a35474c48dcbdf6aba115a86f62/users/96c63fe260a34048961a90001052e647/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:57.284352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b67c1-517c-4d8d-87b8-a59cb1f02a5c None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:57.284645 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b67c1-517c-4d8d-87b8-a59cb1f02a5c None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:57.284913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b67c1-517c-4d8d-87b8-a59cb1f02a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:57.290657 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:57.316699 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-637b67c1-517c-4d8d-87b8-a59cb1f02a5c None None] MFA Rules not processed for user `96c63fe260a34048961a90001052e647`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:57.317956 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-637b67c1-517c-4d8d-87b8-a59cb1f02a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:57.384403 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1434/2852] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:57.396335 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9183da-68a1-4be6-94fd-feadeb09e6fa admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:57.398055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-fb9183da-68a1-4be6-94fd-feadeb09e6fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:57.455885 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9183da-68a1-4be6-94fd-feadeb09e6fa admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:57.459435 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a9da0-727e-4a6f-b289-e728dd9ae583 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c92d7ec81ddc4da7a33f9dbb756e59b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:57.460086 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a9da0-727e-4a6f-b289-e728dd9ae583 service cinder] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:57.460379 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a9da0-727e-4a6f-b289-e728dd9ae583 service cinder] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:57.460659 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a9da0-727e-4a6f-b289-e728dd9ae583 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:57.469385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a9da0-727e-4a6f-b289-e728dd9ae583 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:57.470821 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a9da0-727e-4a6f-b289-e728dd9ae583 service cinder] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:57.479561 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1419/2853] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:42:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:42:57.504254 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b67c1-517c-4d8d-87b8-a59cb1f02a5c None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:57.565539 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b67c1-517c-4d8d-87b8-a59cb1f02a5c None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:57.570128 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e5371-1ee8-4e2d-9e9a-662eedbe64d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:57.570993 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e5371-1ee8-4e2d-9e9a-662eedbe64d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:57.571279 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e5371-1ee8-4e2d-9e9a-662eedbe64d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:57.571582 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e5371-1ee8-4e2d-9e9a-662eedbe64d5 admin admin] PATH_INFO: `/v3/users/96c63fe260a34048961a90001052e647` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:57.572788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e5371-1ee8-4e2d-9e9a-662eedbe64d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96c63fe260a34048961a90001052e647)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:57.575368 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e5371-1ee8-4e2d-9e9a-662eedbe64d5 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:57.625812 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-420e5371-1ee8-4e2d-9e9a-662eedbe64d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96c63fe260a34048961a90001052e647'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:57.626500 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-420e5371-1ee8-4e2d-9e9a-662eedbe64d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96c63fe260a34048961a90001052e647'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:57.638355 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-420e5371-1ee8-4e2d-9e9a-662eedbe64d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96c63fe260a34048961a90001052e647'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:57.639002 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-420e5371-1ee8-4e2d-9e9a-662eedbe64d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96c63fe260a34048961a90001052e647'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:57.646926 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-420e5371-1ee8-4e2d-9e9a-662eedbe64d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96c63fe260a34048961a90001052e647'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:57.660411 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1435/2854] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:42:57 2022] DELETE /identity/v3/users/96c63fe260a34048961a90001052e647 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:57.665050 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e037c-a2bd-4833-8311-bf6aa6c92220 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:57.665387 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e037c-a2bd-4833-8311-bf6aa6c92220 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:57.665673 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e037c-a2bd-4833-8311-bf6aa6c92220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:57.683884 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-040e037c-a2bd-4833-8311-bf6aa6c92220 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:57.685087 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-040e037c-a2bd-4833-8311-bf6aa6c92220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:57.750575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1420/2855] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:57.759649 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e5371-1ee8-4e2d-9e9a-662eedbe64d5 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:57.761057 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-420e5371-1ee8-4e2d-9e9a-662eedbe64d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:57.774809 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:57.821776 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e5371-1ee8-4e2d-9e9a-662eedbe64d5 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:57.826445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9472b-4840-4ec8-bb6e-534eceb649f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:57.827095 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9472b-4840-4ec8-bb6e-534eceb649f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:57.827379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9472b-4840-4ec8-bb6e-534eceb649f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:57.827654 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9472b-4840-4ec8-bb6e-534eceb649f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:57.837146 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9472b-4840-4ec8-bb6e-534eceb649f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:57.838640 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9472b-4840-4ec8-bb6e-534eceb649f9 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:57.849029 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1436/2856] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:42:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:42:58.378452 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e037c-a2bd-4833-8311-bf6aa6c92220 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:58.379884 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-040e037c-a2bd-4833-8311-bf6aa6c92220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:58.394689 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e037c-a2bd-4833-8311-bf6aa6c92220 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:58.399723 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271031c-93c0-4a11-bc2a-29d6ffd050fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:58.400603 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271031c-93c0-4a11-bc2a-29d6ffd050fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:58.400890 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271031c-93c0-4a11-bc2a-29d6ffd050fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:58.401166 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271031c-93c0-4a11-bc2a-29d6ffd050fb admin admin] PATH_INFO: `/v3/projects/dd4f2a35474c48dcbdf6aba115a86f62` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:58.402993 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271031c-93c0-4a11-bc2a-29d6ffd050fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd4f2a35474c48dcbdf6aba115a86f62)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:58.404917 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271031c-93c0-4a11-bc2a-29d6ffd050fb admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:58.443802 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:58.454187 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c271031c-93c0-4a11-bc2a-29d6ffd050fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd4f2a35474c48dcbdf6aba115a86f62', 'request_id': 'req-c271031c-93c0-4a11-bc2a-29d6ffd050fb'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:58.461015 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c271031c-93c0-4a11-bc2a-29d6ffd050fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd4f2a35474c48dcbdf6aba115a86f62', 'request_id': 'req-c271031c-93c0-4a11-bc2a-29d6ffd050fb'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:58.466646 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c271031c-93c0-4a11-bc2a-29d6ffd050fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd4f2a35474c48dcbdf6aba115a86f62', 'request_id': 'req-c271031c-93c0-4a11-bc2a-29d6ffd050fb'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:58.477755 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c271031c-93c0-4a11-bc2a-29d6ffd050fb admin admin] The token cache is being invalidate because project dd4f2a35474c48dcbdf6aba115a86f62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:42:58.478040 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c271031c-93c0-4a11-bc2a-29d6ffd050fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:58.478700 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-c271031c-93c0-4a11-bc2a-29d6ffd050fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:42:58.480780 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1421/2857] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:42:58 2022] DELETE /identity/v3/projects/dd4f2a35474c48dcbdf6aba115a86f62 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:42:58.486306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e3bd1-8c62-4795-a3ed-1bc28792e4a1 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:58.486600 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e3bd1-8c62-4795-a3ed-1bc28792e4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:58.486891 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e3bd1-8c62-4795-a3ed-1bc28792e4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:58.524767 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-705e3bd1-8c62-4795-a3ed-1bc28792e4a1 None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:58.526004 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-705e3bd1-8c62-4795-a3ed-1bc28792e4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:58.586275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1437/2858] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:58.595247 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaad25-dfa2-4f99-b9e1-b2092b5dc4aa None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:58.595617 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaad25-dfa2-4f99-b9e1-b2092b5dc4aa None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:58.595909 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaad25-dfa2-4f99-b9e1-b2092b5dc4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:58.614762 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-0ecaad25-dfa2-4f99-b9e1-b2092b5dc4aa None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:58.616001 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-0ecaad25-dfa2-4f99-b9e1-b2092b5dc4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:58.624673 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1422/2859] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:58.628937 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e3bd1-8c62-4795-a3ed-1bc28792e4a1 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:58.638755 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e3bd1-8c62-4795-a3ed-1bc28792e4a1 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:58.643333 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569cdcd7-8caa-4a43-ad20-f63edde75bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:58.644021 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cdcd7-8caa-4a43-ad20-f63edde75bfc admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:58.644310 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cdcd7-8caa-4a43-ad20-f63edde75bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:58.644595 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cdcd7-8caa-4a43-ad20-f63edde75bfc admin admin] PATH_INFO: `/v3/domains` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:58.645357 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569cdcd7-8caa-4a43-ad20-f63edde75bfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 06:42:58.645672 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569cdcd7-8caa-4a43-ad20-f63edde75bfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:58.647311 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569cdcd7-8caa-4a43-ad20-f63edde75bfc admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:58.659200 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1438/2860] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 06:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 06:42:58.663408 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaad25-dfa2-4f99-b9e1-b2092b5dc4aa None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:58.673382 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaad25-dfa2-4f99-b9e1-b2092b5dc4aa None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:58.678050 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f2bd7-7b9d-4864-9658-2cf15968c1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:58.678700 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f2bd7-7b9d-4864-9658-2cf15968c1a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:58.678987 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f2bd7-7b9d-4864-9658-2cf15968c1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:58.679267 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f2bd7-7b9d-4864-9658-2cf15968c1a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:58.679956 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f2bd7-7b9d-4864-9658-2cf15968c1a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:58.681896 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f2bd7-7b9d-4864-9658-2cf15968c1a4 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:58.711383 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1423/2861] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:42:58 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:42:58.716203 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569cdcd7-8caa-4a43-ad20-f63edde75bfc admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:58.726295 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569cdcd7-8caa-4a43-ad20-f63edde75bfc admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:58.782162 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc91104-7ef0-470f-8ab7-bfdbc209d134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:58.782833 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc91104-7ef0-470f-8ab7-bfdbc209d134 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:58.783119 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc91104-7ef0-470f-8ab7-bfdbc209d134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:58.783399 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc91104-7ef0-470f-8ab7-bfdbc209d134 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:58.784349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc91104-7ef0-470f-8ab7-bfdbc209d134 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:58.786293 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc91104-7ef0-470f-8ab7-bfdbc209d134 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:58.806608 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1439/2862] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:42:58 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:42:58.810781 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f2bd7-7b9d-4864-9658-2cf15968c1a4 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:58.821180 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f2bd7-7b9d-4864-9658-2cf15968c1a4 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:58.825949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec68868-e056-4f79-b8f5-74b1c4ba3f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:58.826608 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec68868-e056-4f79-b8f5-74b1c4ba3f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:58.826893 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec68868-e056-4f79-b8f5-74b1c4ba3f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:58.827170 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec68868-e056-4f79-b8f5-74b1c4ba3f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:58.827906 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec68868-e056-4f79-b8f5-74b1c4ba3f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:58.829509 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec68868-e056-4f79-b8f5-74b1c4ba3f73 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:58.840923 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1424/2863] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:42:58.844896 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc91104-7ef0-470f-8ab7-bfdbc209d134 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:58.854835 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc91104-7ef0-470f-8ab7-bfdbc209d134 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:58.859864 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9551ae7-2d54-4a5a-94ca-266e27a691e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:58.860528 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9551ae7-2d54-4a5a-94ca-266e27a691e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:58.860808 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9551ae7-2d54-4a5a-94ca-266e27a691e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:58.861077 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9551ae7-2d54-4a5a-94ca-266e27a691e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:58.861836 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9551ae7-2d54-4a5a-94ca-266e27a691e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:58.863423 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9551ae7-2d54-4a5a-94ca-266e27a691e4 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:58.873449 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1440/2864] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:42:58.877634 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec68868-e056-4f79-b8f5-74b1c4ba3f73 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:58.887550 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec68868-e056-4f79-b8f5-74b1c4ba3f73 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:58.892115 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7508c8a-8fa1-4e03-a3f0-7ebd24bb33e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:58.892982 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7508c8a-8fa1-4e03-a3f0-7ebd24bb33e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:58.893269 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7508c8a-8fa1-4e03-a3f0-7ebd24bb33e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:58.893657 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7508c8a-8fa1-4e03-a3f0-7ebd24bb33e1 admin admin] PATH_INFO: `/v3/projects/4acaeb032d8e44eb97a4b0a573e7c360/users/697d433699cc475d91609e010917ca16/roles/ed6cf002b70f422190946ebcb114da60` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:58.909533 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7508c8a-8fa1-4e03-a3f0-7ebd24bb33e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4acaeb032d8e44eb97a4b0a573e7c360, user_id=697d433699cc475d91609e010917ca16, role_id=ed6cf002b70f422190946ebcb114da60)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:58.912102 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7508c8a-8fa1-4e03-a3f0-7ebd24bb33e1 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:58.925542 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1425/2865] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:42:58 2022] PUT /identity/v3/projects/4acaeb032d8e44eb97a4b0a573e7c360/users/697d433699cc475d91609e010917ca16/roles/ed6cf002b70f422190946ebcb114da60 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:42:58.930728 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7ab16-709c-4c1c-9018-387b9bbeee2d None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:58.931023 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7ab16-709c-4c1c-9018-387b9bbeee2d None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:58.931306 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7ab16-709c-4c1c-9018-387b9bbeee2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:58.937193 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:42:58.950809 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-58e7ab16-709c-4c1c-9018-387b9bbeee2d None None] MFA Rules not processed for user `5a11fde32c844e228722284109f82f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:42:58.952129 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-58e7ab16-709c-4c1c-9018-387b9bbeee2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:59.016924 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1441/2866] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:42:59.026502 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7508c8a-8fa1-4e03-a3f0-7ebd24bb33e1 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:42:59.028873 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c7508c8a-8fa1-4e03-a3f0-7ebd24bb33e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:42:59.088126 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7508c8a-8fa1-4e03-a3f0-7ebd24bb33e1 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:42:59.091949 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651bb1b9-dc87-4173-bb9d-95e25f6b5c0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:42:59.092595 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bb1b9-dc87-4173-bb9d-95e25f6b5c0e service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:42:59.092882 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bb1b9-dc87-4173-bb9d-95e25f6b5c0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:42:59.093163 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bb1b9-dc87-4173-bb9d-95e25f6b5c0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:42:59.103898 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651bb1b9-dc87-4173-bb9d-95e25f6b5c0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:42:59.106002 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651bb1b9-dc87-4173-bb9d-95e25f6b5c0e service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:42:59.115478 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1426/2867] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:42:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 06:43:01.875790 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16f016-d96b-4e56-9a53-a1448f4de019 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:43:01.876338 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16f016-d96b-4e56-9a53-a1448f4de019 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:43:01.876576 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16f016-d96b-4e56-9a53-a1448f4de019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:43:01.911788 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-ba16f016-d96b-4e56-9a53-a1448f4de019 None None] MFA Rules not processed for user `c1815cc5a2fd43259c5b53650dbc7603`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:43:01.918562 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-ba16f016-d96b-4e56-9a53-a1448f4de019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:43:02.011831 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1442/2868] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:43:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:43:02.588204 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651bb1b9-dc87-4173-bb9d-95e25f6b5c0e service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:43:02.590424 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-651bb1b9-dc87-4173-bb9d-95e25f6b5c0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:43:02.603064 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651bb1b9-dc87-4173-bb9d-95e25f6b5c0e service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:43:02.608427 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca3a63-a94c-4d72-934b-079de27dceae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:43:02.609264 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca3a63-a94c-4d72-934b-079de27dceae service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:43:02.609822 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca3a63-a94c-4d72-934b-079de27dceae service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:43:02.610253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca3a63-a94c-4d72-934b-079de27dceae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:43:02.613010 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-bbca3a63-a94c-4d72-934b-079de27dceae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:43:02.626262 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca3a63-a94c-4d72-934b-079de27dceae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:43:02.627897 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca3a63-a94c-4d72-934b-079de27dceae service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:43:02.638931 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1427/2869] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:43:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:43:03.720257 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c28e3-01fd-4d0f-9a1b-0d67893bee12 None None] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:43:03.720760 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c28e3-01fd-4d0f-9a1b-0d67893bee12 None None] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:43:03.720960 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c28e3-01fd-4d0f-9a1b-0d67893bee12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:43:03.763359 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.auth.core [None req-679c28e3-01fd-4d0f-9a1b-0d67893bee12 None None] MFA Rules not processed for user `697d433699cc475d91609e010917ca16`. Rule list: `[]` (Enabled: `True`). {{(pid=80280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:43:03.780712 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-679c28e3-01fd-4d0f-9a1b-0d67893bee12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:43:03.920796 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1443/2870] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:43:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:43:03.926803 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca3a63-a94c-4d72-934b-079de27dceae service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:43:03.985658 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca3a63-a94c-4d72-934b-079de27dceae service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:43:04.001849 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec73d77-8fbb-4294-bf6d-48550302b958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:43:04.002684 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec73d77-8fbb-4294-bf6d-48550302b958 admin admin] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:43:04.004373 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec73d77-8fbb-4294-bf6d-48550302b958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:43:04.004891 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec73d77-8fbb-4294-bf6d-48550302b958 admin admin] PATH_INFO: `/v3/projects` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:43:04.006232 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec73d77-8fbb-4294-bf6d-48550302b958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:43:04.010583 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec73d77-8fbb-4294-bf6d-48550302b958 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:43:04.078869 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1428/2871] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 06:43:03 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 06:43:04.084656 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c28e3-01fd-4d0f-9a1b-0d67893bee12 None None] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:43:04.101275 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c28e3-01fd-4d0f-9a1b-0d67893bee12 None None] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:43:04.146491 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=100 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:43:04.177531 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e45be9-06f0-4f0b-b3e3-298bbf29c566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:43:04.178311 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e45be9-06f0-4f0b-b3e3-298bbf29c566 admin admin] REQUEST_METHOD: `POST` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:43:04.178656 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e45be9-06f0-4f0b-b3e3-298bbf29c566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:43:04.178999 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e45be9-06f0-4f0b-b3e3-298bbf29c566 admin admin] PATH_INFO: `/v3/users` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:43:04.179781 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e45be9-06f0-4f0b-b3e3-298bbf29c566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:43:04.183476 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e45be9-06f0-4f0b-b3e3-298bbf29c566 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:43:04.213890 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1444/2872] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 06:43:04 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 06:43:04.218348 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec73d77-8fbb-4294-bf6d-48550302b958 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:43:04.230830 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec73d77-8fbb-4294-bf6d-48550302b958 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:43:04.235915 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b5991-4002-4e5a-a26a-e003b9036523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:43:04.236600 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b5991-4002-4e5a-a26a-e003b9036523 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:43:04.236889 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b5991-4002-4e5a-a26a-e003b9036523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:43:04.237168 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b5991-4002-4e5a-a26a-e003b9036523 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:43:04.238037 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b5991-4002-4e5a-a26a-e003b9036523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:43:04.239546 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b5991-4002-4e5a-a26a-e003b9036523 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:43:04.252449 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1429/2873] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 06:43:04.257704 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e45be9-06f0-4f0b-b3e3-298bbf29c566 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:43:04.269269 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e45be9-06f0-4f0b-b3e3-298bbf29c566 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:43:04.274879 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fb69b-de48-4eb4-b793-4ff89c67110a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:43:04.275981 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fb69b-de48-4eb4-b793-4ff89c67110a admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:43:04.276281 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fb69b-de48-4eb4-b793-4ff89c67110a admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:43:04.277147 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fb69b-de48-4eb4-b793-4ff89c67110a admin admin] PATH_INFO: `/v3/projects/a5bb62a5058b443badb95969560fd41b/users/28560d1cc848407bbb2545f324a1aacc/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:43:04.298930 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fb69b-de48-4eb4-b793-4ff89c67110a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5bb62a5058b443badb95969560fd41b, user_id=28560d1cc848407bbb2545f324a1aacc, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:43:04.302088 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fb69b-de48-4eb4-b793-4ff89c67110a admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:43:04.328207 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1445/2874] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:43:04 2022] PUT /identity/v3/projects/a5bb62a5058b443badb95969560fd41b/users/28560d1cc848407bbb2545f324a1aacc/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:43:04.332449 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b5991-4002-4e5a-a26a-e003b9036523 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:43:04.346103 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b5991-4002-4e5a-a26a-e003b9036523 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:43:04.363466 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:43:04.427142 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38b28d-f268-48dd-96f0-30a02780d5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:43:04.427828 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b28d-f268-48dd-96f0-30a02780d5b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:43:04.428113 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b28d-f268-48dd-96f0-30a02780d5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:43:04.428396 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b28d-f268-48dd-96f0-30a02780d5b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:43:04.429352 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38b28d-f268-48dd-96f0-30a02780d5b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:43:04.431011 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38b28d-f268-48dd-96f0-30a02780d5b6 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:43:04.444018 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1430/2875] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 06:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 06:43:04.449210 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fb69b-de48-4eb4-b793-4ff89c67110a admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:43:04.469349 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fb69b-de48-4eb4-b793-4ff89c67110a admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:43:04.474371 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb46a0-813d-4311-b646-a8b4c6c33983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:43:04.475274 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb46a0-813d-4311-b646-a8b4c6c33983 admin admin] REQUEST_METHOD: `PUT` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:43:04.476538 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb46a0-813d-4311-b646-a8b4c6c33983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:43:04.476827 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb46a0-813d-4311-b646-a8b4c6c33983 admin admin] PATH_INFO: `/v3/projects/a5bb62a5058b443badb95969560fd41b/users/28560d1cc848407bbb2545f324a1aacc/roles/e0653fd3bcd64446979a2a9b403de431` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:43:04.479277 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb46a0-813d-4311-b646-a8b4c6c33983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5bb62a5058b443badb95969560fd41b, user_id=28560d1cc848407bbb2545f324a1aacc, role_id=e0653fd3bcd64446979a2a9b403de431)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:43:04.483090 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb46a0-813d-4311-b646-a8b4c6c33983 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:43:04.496445 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1446/2876] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 06:43:04 2022] PUT /identity/v3/projects/a5bb62a5058b443badb95969560fd41b/users/28560d1cc848407bbb2545f324a1aacc/roles/e0653fd3bcd64446979a2a9b403de431 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 06:43:09.896745 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7621a9b-f47f-4a68-8cd3-7e560a7f0ff5 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:43:09.897387 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7621a9b-f47f-4a68-8cd3-7e560a7f0ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:43:09.897387 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7621a9b-f47f-4a68-8cd3-7e560a7f0ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:43:09.928236 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-c7621a9b-f47f-4a68-8cd3-7e560a7f0ff5 None None] MFA Rules not processed for user `28560d1cc848407bbb2545f324a1aacc`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:43:09.929687 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-c7621a9b-f47f-4a68-8cd3-7e560a7f0ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:43:10.010244 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1431/2877] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:43:09 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:43:10.025535 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb46a0-813d-4311-b646-a8b4c6c33983 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:43:10.027312 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0cbb46a0-813d-4311-b646-a8b4c6c33983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:43:10.111475 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb46a0-813d-4311-b646-a8b4c6c33983 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:43:10.117046 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809c33b-6cc2-4153-907d-1bc04521f5f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:43:10.117751 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809c33b-6cc2-4153-907d-1bc04521f5f5 service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:43:10.118039 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809c33b-6cc2-4153-907d-1bc04521f5f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:43:10.118324 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809c33b-6cc2-4153-907d-1bc04521f5f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:43:10.178536 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809c33b-6cc2-4153-907d-1bc04521f5f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:43:10.180068 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809c33b-6cc2-4153-907d-1bc04521f5f5 service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:43:10.192945 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1447/2878] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:43:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:43:11.266250 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85c05e-9279-4037-8bc5-0099a02e9fe5 None None] REQUEST_METHOD: `POST` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:43:11.266778 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85c05e-9279-4037-8bc5-0099a02e9fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:43:11.266882 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85c05e-9279-4037-8bc5-0099a02e9fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:43:11.285983 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.auth.core [None req-4b85c05e-9279-4037-8bc5-0099a02e9fe5 None None] MFA Rules not processed for user `697d433699cc475d91609e010917ca16`. Rule list: `[]` (Enabled: `True`). {{(pid=80281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 06:43:11.287221 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4b85c05e-9279-4037-8bc5-0099a02e9fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:43:11.295456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1432/2879] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 06:43:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 06:43:18.240404 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809c33b-6cc2-4153-907d-1bc04521f5f5 service nova] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:43:18.256206 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809c33b-6cc2-4153-907d-1bc04521f5f5 service nova] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:43:18.263912 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767c955-4f31-4a9d-9f19-b74500a5930b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:43:18.264717 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767c955-4f31-4a9d-9f19-b74500a5930b service nova] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:43:18.264976 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767c955-4f31-4a9d-9f19-b74500a5930b service nova] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:43:18.265344 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767c955-4f31-4a9d-9f19-b74500a5930b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:43:18.267105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-a767c955-4f31-4a9d-9f19-b74500a5930b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:43:18.281616 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767c955-4f31-4a9d-9f19-b74500a5930b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:43:18.283615 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767c955-4f31-4a9d-9f19-b74500a5930b service nova] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:43:18.297272 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1448/2880] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:43:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:43:21.280694 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=58 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:43:28.291696 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=89,keystone:INSERT=5 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:43:42.358672 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85c05e-9279-4037-8bc5-0099a02e9fe5 None None] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:43:42.360079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4b85c05e-9279-4037-8bc5-0099a02e9fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:43:42.451385 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85c05e-9279-4037-8bc5-0099a02e9fe5 None None] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:43:42.456483 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07098081-4a0d-4cad-a06d-662dfcd70750 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b76876d821e4bbbb8d5cad5bd26f559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:43:42.457140 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07098081-4a0d-4cad-a06d-662dfcd70750 service placement] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:43:42.457467 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07098081-4a0d-4cad-a06d-662dfcd70750 service placement] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:43:42.457752 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07098081-4a0d-4cad-a06d-662dfcd70750 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:43:42.459534 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-07098081-4a0d-4cad-a06d-662dfcd70750 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:43:42.471217 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07098081-4a0d-4cad-a06d-662dfcd70750 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:43:42.472817 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07098081-4a0d-4cad-a06d-662dfcd70750 service placement] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:43:42.483311 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1433/2881] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:43:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:43:42.815362 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767c955-4f31-4a9d-9f19-b74500a5930b service nova] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:43:42.870542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767c955-4f31-4a9d-9f19-b74500a5930b service nova] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:43:42.874162 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71599e3-d542-4db5-ba1e-f7fdff45ee37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:43:42.874816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71599e3-d542-4db5-ba1e-f7fdff45ee37 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:43:42.875105 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71599e3-d542-4db5-ba1e-f7fdff45ee37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:43:42.875389 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71599e3-d542-4db5-ba1e-f7fdff45ee37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:43:42.885682 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71599e3-d542-4db5-ba1e-f7fdff45ee37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:43:42.887140 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71599e3-d542-4db5-ba1e-f7fdff45ee37 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:43:42.897402 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1449/2882] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:43:42 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:43:44.500128 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07098081-4a0d-4cad-a06d-662dfcd70750 service placement] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:43:44.520397 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07098081-4a0d-4cad-a06d-662dfcd70750 service placement] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:43:44.525859 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142492de-0bfd-4090-ad8f-15cd741d37dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:43:44.526508 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142492de-0bfd-4090-ad8f-15cd741d37dc service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:43:44.526792 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142492de-0bfd-4090-ad8f-15cd741d37dc service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:43:44.527069 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142492de-0bfd-4090-ad8f-15cd741d37dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:43:44.528825 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-142492de-0bfd-4090-ad8f-15cd741d37dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:43:44.542545 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142492de-0bfd-4090-ad8f-15cd741d37dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:43:44.544132 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142492de-0bfd-4090-ad8f-15cd741d37dc service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:43:44.556013 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1434/2883] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:43:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:43:52.892413 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=26 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:43:54.551055 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=50 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:45:01.948453 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71599e3-d542-4db5-ba1e-f7fdff45ee37 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:45:01.957519 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71599e3-d542-4db5-ba1e-f7fdff45ee37 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:45:01.961110 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef06151-d124-482c-9864-7b45f92680f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:45:01.961787 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef06151-d124-482c-9864-7b45f92680f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:45:01.962235 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef06151-d124-482c-9864-7b45f92680f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:45:01.962520 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef06151-d124-482c-9864-7b45f92680f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:45:01.964313 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6ef06151-d124-482c-9864-7b45f92680f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:45:02.023222 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef06151-d124-482c-9864-7b45f92680f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:45:02.024782 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef06151-d124-482c-9864-7b45f92680f0 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:45:02.033873 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1450/2884] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:45:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:45:12.029697 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=28 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:45:15.615021 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142492de-0bfd-4090-ad8f-15cd741d37dc service nova] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:45:15.625253 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142492de-0bfd-4090-ad8f-15cd741d37dc service nova] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:45:15.628838 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159e42f-3552-4ecc-9bdc-f4b2d48a7a0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:45:15.629514 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159e42f-3552-4ecc-9bdc-f4b2d48a7a0e service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:45:15.629802 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159e42f-3552-4ecc-9bdc-f4b2d48a7a0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:45:15.630081 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159e42f-3552-4ecc-9bdc-f4b2d48a7a0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:45:15.631852 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-4159e42f-3552-4ecc-9bdc-f4b2d48a7a0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:45:15.696076 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159e42f-3552-4ecc-9bdc-f4b2d48a7a0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:45:15.697810 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159e42f-3552-4ecc-9bdc-f4b2d48a7a0e service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:45:15.707455 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1435/2885] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:45:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:45:22.021352 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef06151-d124-482c-9864-7b45f92680f0 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:45:22.022868 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-6ef06151-d124-482c-9864-7b45f92680f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:45:22.039644 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef06151-d124-482c-9864-7b45f92680f0 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:45:22.044793 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff4e70-bde6-410e-b1b4-100c28b6a80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:45:22.045686 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff4e70-bde6-410e-b1b4-100c28b6a80f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:45:22.045972 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff4e70-bde6-410e-b1b4-100c28b6a80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:45:22.046251 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff4e70-bde6-410e-b1b4-100c28b6a80f admin admin] PATH_INFO: `/v3/users/c1815cc5a2fd43259c5b53650dbc7603` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:45:22.047450 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff4e70-bde6-410e-b1b4-100c28b6a80f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1815cc5a2fd43259c5b53650dbc7603)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:45:22.049387 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff4e70-bde6-410e-b1b4-100c28b6a80f admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:45:22.096455 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b2ff4e70-bde6-410e-b1b4-100c28b6a80f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1815cc5a2fd43259c5b53650dbc7603'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.097069 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b2ff4e70-bde6-410e-b1b4-100c28b6a80f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1815cc5a2fd43259c5b53650dbc7603'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.109681 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b2ff4e70-bde6-410e-b1b4-100c28b6a80f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1815cc5a2fd43259c5b53650dbc7603'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.110440 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b2ff4e70-bde6-410e-b1b4-100c28b6a80f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1815cc5a2fd43259c5b53650dbc7603'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.118629 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-b2ff4e70-bde6-410e-b1b4-100c28b6a80f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1815cc5a2fd43259c5b53650dbc7603'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.131559 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1451/2886] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:45:22 2022] DELETE /identity/v3/users/c1815cc5a2fd43259c5b53650dbc7603 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:45:22.304560 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159e42f-3552-4ecc-9bdc-f4b2d48a7a0e service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:45:22.320093 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159e42f-3552-4ecc-9bdc-f4b2d48a7a0e service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:45:22.384578 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a82edc-afb6-4fb5-a2fc-d57448bb894c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:45:22.386174 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a82edc-afb6-4fb5-a2fc-d57448bb894c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:45:22.386464 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a82edc-afb6-4fb5-a2fc-d57448bb894c admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:45:22.386741 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a82edc-afb6-4fb5-a2fc-d57448bb894c admin admin] PATH_INFO: `/v3/projects/318e4e0d9f3a49f69eb3d846539b6b67` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:45:22.387924 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a82edc-afb6-4fb5-a2fc-d57448bb894c admin admin] RBAC: Authorizing `identity:delete_project(project_id=318e4e0d9f3a49f69eb3d846539b6b67)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:45:22.389867 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a82edc-afb6-4fb5-a2fc-d57448bb894c admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:45:22.436821 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-76a82edc-afb6-4fb5-a2fc-d57448bb894c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '318e4e0d9f3a49f69eb3d846539b6b67', 'request_id': 'req-76a82edc-afb6-4fb5-a2fc-d57448bb894c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.444019 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-76a82edc-afb6-4fb5-a2fc-d57448bb894c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '318e4e0d9f3a49f69eb3d846539b6b67', 'request_id': 'req-76a82edc-afb6-4fb5-a2fc-d57448bb894c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.450238 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-76a82edc-afb6-4fb5-a2fc-d57448bb894c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '318e4e0d9f3a49f69eb3d846539b6b67', 'request_id': 'req-76a82edc-afb6-4fb5-a2fc-d57448bb894c'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.461231 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-76a82edc-afb6-4fb5-a2fc-d57448bb894c admin admin] The token cache is being invalidate because project 318e4e0d9f3a49f69eb3d846539b6b67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:45:22.461560 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-76a82edc-afb6-4fb5-a2fc-d57448bb894c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.462207 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-76a82edc-afb6-4fb5-a2fc-d57448bb894c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.464390 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1436/2887] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:45:22 2022] DELETE /identity/v3/projects/318e4e0d9f3a49f69eb3d846539b6b67 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:45:22.468429 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff4e70-bde6-410e-b1b4-100c28b6a80f admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:45:22.472494 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-b2ff4e70-bde6-410e-b1b4-100c28b6a80f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:45:22.483450 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff4e70-bde6-410e-b1b4-100c28b6a80f admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:45:22.546119 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fba499-7576-468b-972d-5d64a29823ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:45:22.547009 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fba499-7576-468b-972d-5d64a29823ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:45:22.547297 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fba499-7576-468b-972d-5d64a29823ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:45:22.547586 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fba499-7576-468b-972d-5d64a29823ff admin admin] PATH_INFO: `/v3/users/bd6c775c3efc49cd8f0000f57f96a24b` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:45:22.548816 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fba499-7576-468b-972d-5d64a29823ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd6c775c3efc49cd8f0000f57f96a24b)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:45:22.550780 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fba499-7576-468b-972d-5d64a29823ff admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:45:22.598361 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-25fba499-7576-468b-972d-5d64a29823ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd6c775c3efc49cd8f0000f57f96a24b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.599018 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-25fba499-7576-468b-972d-5d64a29823ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd6c775c3efc49cd8f0000f57f96a24b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.611234 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-25fba499-7576-468b-972d-5d64a29823ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd6c775c3efc49cd8f0000f57f96a24b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.611866 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-25fba499-7576-468b-972d-5d64a29823ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd6c775c3efc49cd8f0000f57f96a24b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.619018 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-25fba499-7576-468b-972d-5d64a29823ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd6c775c3efc49cd8f0000f57f96a24b'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.621680 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:45:22.634185 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1452/2888] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:45:22 2022] DELETE /identity/v3/users/bd6c775c3efc49cd8f0000f57f96a24b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:45:22.819998 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a82edc-afb6-4fb5-a2fc-d57448bb894c admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:45:22.821523 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-76a82edc-afb6-4fb5-a2fc-d57448bb894c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:45:22.835134 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a82edc-afb6-4fb5-a2fc-d57448bb894c admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:45:22.856960 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:45:22.898353 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c9e4c-468e-4bd2-9764-d73fb687ab37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:45:22.899171 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c9e4c-468e-4bd2-9764-d73fb687ab37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:45:22.899443 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c9e4c-468e-4bd2-9764-d73fb687ab37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:45:22.899725 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c9e4c-468e-4bd2-9764-d73fb687ab37 admin admin] PATH_INFO: `/v3/projects/8aaacebf8e2d4c219ad3adf708ac7dd5` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:45:22.901005 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c9e4c-468e-4bd2-9764-d73fb687ab37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aaacebf8e2d4c219ad3adf708ac7dd5)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:45:22.902986 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c9e4c-468e-4bd2-9764-d73fb687ab37 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:45:22.953570 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-de1c9e4c-468e-4bd2-9764-d73fb687ab37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aaacebf8e2d4c219ad3adf708ac7dd5', 'request_id': 'req-de1c9e4c-468e-4bd2-9764-d73fb687ab37'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.961517 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-de1c9e4c-468e-4bd2-9764-d73fb687ab37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aaacebf8e2d4c219ad3adf708ac7dd5', 'request_id': 'req-de1c9e4c-468e-4bd2-9764-d73fb687ab37'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.967456 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-de1c9e4c-468e-4bd2-9764-d73fb687ab37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aaacebf8e2d4c219ad3adf708ac7dd5', 'request_id': 'req-de1c9e4c-468e-4bd2-9764-d73fb687ab37'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.978491 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-de1c9e4c-468e-4bd2-9764-d73fb687ab37 admin admin] The token cache is being invalidate because project 8aaacebf8e2d4c219ad3adf708ac7dd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:45:22.978783 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-de1c9e4c-468e-4bd2-9764-d73fb687ab37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.979647 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-de1c9e4c-468e-4bd2-9764-d73fb687ab37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:45:22.981246 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1437/2889] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:45:22 2022] DELETE /identity/v3/projects/8aaacebf8e2d4c219ad3adf708ac7dd5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:45:32.633299 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=4 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:45:32.975575 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:45:39.757440 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fba499-7576-468b-972d-5d64a29823ff admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:45:39.758975 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-25fba499-7576-468b-972d-5d64a29823ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:45:39.817887 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fba499-7576-468b-972d-5d64a29823ff admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:45:39.866104 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa870e2-47ce-4fc9-adaa-af78b110a6ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:45:39.866758 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa870e2-47ce-4fc9-adaa-af78b110a6ce service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:45:39.867045 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa870e2-47ce-4fc9-adaa-af78b110a6ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:45:39.867323 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa870e2-47ce-4fc9-adaa-af78b110a6ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:45:39.869133 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0aa870e2-47ce-4fc9-adaa-af78b110a6ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:45:39.950292 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa870e2-47ce-4fc9-adaa-af78b110a6ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:45:39.951778 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa870e2-47ce-4fc9-adaa-af78b110a6ce service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:45:39.972148 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1453/2890] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:45:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:45:41.157933 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c9e4c-468e-4bd2-9764-d73fb687ab37 admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:45:41.159405 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-de1c9e4c-468e-4bd2-9764-d73fb687ab37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:45:41.172503 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c9e4c-468e-4bd2-9764-d73fb687ab37 admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:45:41.176125 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ec158-75cc-489b-9255-c57f4cd9aabb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:45:41.176768 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ec158-75cc-489b-9255-c57f4cd9aabb service neutron] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:45:41.177054 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ec158-75cc-489b-9255-c57f4cd9aabb service neutron] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:45:41.177368 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ec158-75cc-489b-9255-c57f4cd9aabb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:45:41.179079 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e38ec158-75cc-489b-9255-c57f4cd9aabb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:45:41.193048 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ec158-75cc-489b-9255-c57f4cd9aabb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:45:41.194524 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ec158-75cc-489b-9255-c57f4cd9aabb service neutron] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:45:41.206136 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1438/2891] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:45:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:45:49.969607 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=83 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:45:51.201139 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=6 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:46:51.045983 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa870e2-47ce-4fc9-adaa-af78b110a6ce service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:46:51.054841 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa870e2-47ce-4fc9-adaa-af78b110a6ce service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:46:51.058491 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f620e65-f4ac-40e2-9b30-b89ad0d9aff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '226f4700a516415cbbd272fdfad7af58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:46:51.059134 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f620e65-f4ac-40e2-9b30-b89ad0d9aff8 service neutron] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:46:51.059417 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f620e65-f4ac-40e2-9b30-b89ad0d9aff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:46:51.059691 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f620e65-f4ac-40e2-9b30-b89ad0d9aff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:46:51.061542 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2f620e65-f4ac-40e2-9b30-b89ad0d9aff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:46:51.071588 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f620e65-f4ac-40e2-9b30-b89ad0d9aff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:46:51.073033 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f620e65-f4ac-40e2-9b30-b89ad0d9aff8 service neutron] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:46:51.081146 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1454/2892] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:46:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:47:01.077502 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=6 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:47:16.778083 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ec158-75cc-489b-9255-c57f4cd9aabb service neutron] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:47:16.780319 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-e38ec158-75cc-489b-9255-c57f4cd9aabb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:47:16.796167 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ec158-75cc-489b-9255-c57f4cd9aabb service neutron] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:47:16.803637 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63f525-91a3-4fcd-930c-2c2cf10269fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a09f4878eac4d60a9a7dcedba15c05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:47:16.804297 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63f525-91a3-4fcd-930c-2c2cf10269fd service nova] REQUEST_METHOD: `GET` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:47:16.805579 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63f525-91a3-4fcd-930c-2c2cf10269fd service nova] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:47:16.805870 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63f525-91a3-4fcd-930c-2c2cf10269fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:47:16.808166 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-3c63f525-91a3-4fcd-930c-2c2cf10269fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:47:16.823786 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63f525-91a3-4fcd-930c-2c2cf10269fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:47:16.825845 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63f525-91a3-4fcd-930c-2c2cf10269fd service nova] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:47:16.838827 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1439/2893] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:47:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:47:26.828548 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f620e65-f4ac-40e2-9b30-b89ad0d9aff8 service neutron] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:47:26.833711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=6 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:47:26.836892 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-2f620e65-f4ac-40e2-9b30-b89ad0d9aff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:47:26.913379 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f620e65-f4ac-40e2-9b30-b89ad0d9aff8 service neutron] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:47:26.919614 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f2512-b466-4410-8f18-632588e516a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:47:26.920580 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2512-b466-4410-8f18-632588e516a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:47:26.920870 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2512-b466-4410-8f18-632588e516a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:47:26.921156 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2512-b466-4410-8f18-632588e516a2 admin admin] PATH_INFO: `/v3/users/697d433699cc475d91609e010917ca16` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:47:26.922512 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f2512-b466-4410-8f18-632588e516a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=697d433699cc475d91609e010917ca16)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:47:26.924443 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f2512-b466-4410-8f18-632588e516a2 admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:47:26.984082 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8f5f2512-b466-4410-8f18-632588e516a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '697d433699cc475d91609e010917ca16'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:26.984797 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8f5f2512-b466-4410-8f18-632588e516a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '697d433699cc475d91609e010917ca16'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:26.998880 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8f5f2512-b466-4410-8f18-632588e516a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '697d433699cc475d91609e010917ca16'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:26.999727 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8f5f2512-b466-4410-8f18-632588e516a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '697d433699cc475d91609e010917ca16'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:27.008171 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-8f5f2512-b466-4410-8f18-632588e516a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '697d433699cc475d91609e010917ca16'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:27.020928 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1455/2894] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:47:26 2022] DELETE /identity/v3/users/697d433699cc475d91609e010917ca16 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:47:27.200889 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63f525-91a3-4fcd-930c-2c2cf10269fd service nova] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:47:27.212157 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63f525-91a3-4fcd-930c-2c2cf10269fd service nova] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:47:27.270318 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277ef94-9268-4744-8abb-f9b5fffc302f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:47:27.271095 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277ef94-9268-4744-8abb-f9b5fffc302f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:47:27.271389 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277ef94-9268-4744-8abb-f9b5fffc302f admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:47:27.271669 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277ef94-9268-4744-8abb-f9b5fffc302f admin admin] PATH_INFO: `/v3/projects/4acaeb032d8e44eb97a4b0a573e7c360` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:47:27.272860 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d277ef94-9268-4744-8abb-f9b5fffc302f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4acaeb032d8e44eb97a4b0a573e7c360)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:47:27.275041 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d277ef94-9268-4744-8abb-f9b5fffc302f admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:47:27.324164 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d277ef94-9268-4744-8abb-f9b5fffc302f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4acaeb032d8e44eb97a4b0a573e7c360', 'request_id': 'req-d277ef94-9268-4744-8abb-f9b5fffc302f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:27.332640 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d277ef94-9268-4744-8abb-f9b5fffc302f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4acaeb032d8e44eb97a4b0a573e7c360', 'request_id': 'req-d277ef94-9268-4744-8abb-f9b5fffc302f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:27.338889 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d277ef94-9268-4744-8abb-f9b5fffc302f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4acaeb032d8e44eb97a4b0a573e7c360', 'request_id': 'req-d277ef94-9268-4744-8abb-f9b5fffc302f'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:27.350018 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d277ef94-9268-4744-8abb-f9b5fffc302f admin admin] The token cache is being invalidate because project 4acaeb032d8e44eb97a4b0a573e7c360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:47:27.350306 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d277ef94-9268-4744-8abb-f9b5fffc302f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:27.350962 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-d277ef94-9268-4744-8abb-f9b5fffc302f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:27.353152 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1440/2895] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:47:27 2022] DELETE /identity/v3/projects/4acaeb032d8e44eb97a4b0a573e7c360 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:47:27.358109 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f2512-b466-4410-8f18-632588e516a2 admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:47:27.359590 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-8f5f2512-b466-4410-8f18-632588e516a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:47:27.373203 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f2512-b466-4410-8f18-632588e516a2 admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:47:27.431295 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb4c85-6c43-41b6-9ad7-4718549acdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:47:27.432159 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb4c85-6c43-41b6-9ad7-4718549acdfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:47:27.432447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb4c85-6c43-41b6-9ad7-4718549acdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:47:27.432733 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb4c85-6c43-41b6-9ad7-4718549acdfb admin admin] PATH_INFO: `/v3/users/28560d1cc848407bbb2545f324a1aacc` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:47:27.434855 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb4c85-6c43-41b6-9ad7-4718549acdfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=28560d1cc848407bbb2545f324a1aacc)` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:47:27.437556 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb4c85-6c43-41b6-9ad7-4718549acdfb admin admin] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:47:27.459935 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:47:27.487670 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-fabb4c85-6c43-41b6-9ad7-4718549acdfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28560d1cc848407bbb2545f324a1aacc'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:27.488348 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-fabb4c85-6c43-41b6-9ad7-4718549acdfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28560d1cc848407bbb2545f324a1aacc'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:27.500913 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-fabb4c85-6c43-41b6-9ad7-4718549acdfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28560d1cc848407bbb2545f324a1aacc'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:27.501627 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-fabb4c85-6c43-41b6-9ad7-4718549acdfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28560d1cc848407bbb2545f324a1aacc'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:27.510015 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.notifications [None req-fabb4c85-6c43-41b6-9ad7-4718549acdfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28560d1cc848407bbb2545f324a1aacc'} {{(pid=80280) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:27.522447 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1456/2896] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 06:47:27 2022] DELETE /identity/v3/users/28560d1cc848407bbb2545f324a1aacc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:47:27.705531 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277ef94-9268-4744-8abb-f9b5fffc302f admin admin] Authenticating user token {{(pid=80281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:47:27.707051 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.fernet_utils [None req-d277ef94-9268-4744-8abb-f9b5fffc302f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:47:27.723383 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277ef94-9268-4744-8abb-f9b5fffc302f admin admin] Validating token access rules against request {{(pid=80281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:47:27.786711 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37daf664-520a-4a2c-8fbc-ba95faab4c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a11fde32c844e228722284109f82f80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0443648214584699a9ada0deda814964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:47:27.787540 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37daf664-520a-4a2c-8fbc-ba95faab4c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:47:27.787856 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37daf664-520a-4a2c-8fbc-ba95faab4c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:47:27.788138 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37daf664-520a-4a2c-8fbc-ba95faab4c33 admin admin] PATH_INFO: `/v3/projects/a5bb62a5058b443badb95969560fd41b` {{(pid=80281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:47:27.790201 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37daf664-520a-4a2c-8fbc-ba95faab4c33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5bb62a5058b443badb95969560fd41b)` {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:47:27.792076 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37daf664-520a-4a2c-8fbc-ba95faab4c33 admin admin] RBAC: Authorization granted {{(pid=80281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:47:27.828230 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:47:27.842825 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-37daf664-520a-4a2c-8fbc-ba95faab4c33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5bb62a5058b443badb95969560fd41b', 'request_id': 'req-37daf664-520a-4a2c-8fbc-ba95faab4c33'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:27.850574 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-37daf664-520a-4a2c-8fbc-ba95faab4c33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5bb62a5058b443badb95969560fd41b', 'request_id': 'req-37daf664-520a-4a2c-8fbc-ba95faab4c33'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:27.856975 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-37daf664-520a-4a2c-8fbc-ba95faab4c33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5bb62a5058b443badb95969560fd41b', 'request_id': 'req-37daf664-520a-4a2c-8fbc-ba95faab4c33'} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:27.868324 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-37daf664-520a-4a2c-8fbc-ba95faab4c33 admin admin] The token cache is being invalidate because project a5bb62a5058b443badb95969560fd41b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80281) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 06:47:27.868615 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-37daf664-520a-4a2c-8fbc-ba95faab4c33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:27.869273 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG keystone.notifications [None req-37daf664-520a-4a2c-8fbc-ba95faab4c33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80281) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 06:47:27.870843 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: [pid: 80281|app: 0|req: 1441/2897] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 06:47:27 2022] DELETE /identity/v3/projects/a5bb62a5058b443badb95969560fd41b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 06:47:37.518819 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:47:37.865181 dv-cinder-836996-22-iscsi devstack@keystone.service[80281]: DEBUG dbcounter [-] [80281] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80281) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 06:48:50.286613 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb4c85-6c43-41b6-9ad7-4718549acdfb admin admin] Authenticating user token {{(pid=80280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 06:48:50.288030 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-fabb4c85-6c43-41b6-9ad7-4718549acdfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:48:50.357656 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb4c85-6c43-41b6-9ad7-4718549acdfb admin admin] Validating token access rules against request {{(pid=80280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 06:48:50.418066 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75214a-cd05-4f3d-825b-50f38119c411 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b76876d821e4bbbb8d5cad5bd26f559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a3d399dc95424b848b736c0d0298d9a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 06:48:50.418722 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75214a-cd05-4f3d-825b-50f38119c411 service placement] REQUEST_METHOD: `GET` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 06:48:50.419006 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75214a-cd05-4f3d-825b-50f38119c411 service placement] SCRIPT_NAME: `/identity` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 06:48:50.419282 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75214a-cd05-4f3d-825b-50f38119c411 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 06:48:50.421995 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.fernet_utils [None req-0a75214a-cd05-4f3d-825b-50f38119c411 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 06:48:50.433432 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75214a-cd05-4f3d-825b-50f38119c411 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 06:48:50.435193 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75214a-cd05-4f3d-825b-50f38119c411 service placement] RBAC: Authorization granted {{(pid=80280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 06:48:50.445882 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: [pid: 80280|app: 0|req: 1457/2898] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 06:48:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 06:49:00.441329 dv-cinder-836996-22-iscsi devstack@keystone.service[80280]: DEBUG dbcounter [-] [80280] Writing DB stats keystone:SELECT=58 {{(pid=80280) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}